User Embeddings: Part 2 — Powering Personalization at Scale

In the first part, we explored the fundamentals of user embeddings and how they fuel personalized recommendations. This follow-up dives deeper into the engineering, challenges, and ethics behind real-world systems like TikTok, YouTube, and Netflix.

---

Going Beyond Dot Products: Real-World Architectures

Most real-world recommender systems move far beyond simple matrix factorization. TikTok, for example, uses a multi-stage ranking system:

1. Candidate Generation: Rapidly filters millions of videos to a few thousand using collaborative filtering.
2. Pre-Ranking: Applies user/video embeddings and basic behavioral models.
3. Ranking Model: A deep neural network (DNN) that weighs long-term preferences, recent activity, device context, etc.

These systems optimize for metrics like:

• Watch time
• Engagement rate
• Likelihood to follow/like/share/comment
• Long-term retention

---

Cold-Start Problem and Hybrid Solutions

The Cold Start Problem

New users or new content = no history. Traditional collaborative filtering fails here.

Solutions:

• Demographic-based embeddings (e.g., age, location, device)
• Content-based filtering: Use NLP or vision models on text/images
• Session-based models: Create temporary embeddings from 1–2 interactions
• Meta-learned embeddings: Few-shot learning techniques like MAML or Reptile

---

Deep Learning for Embeddings

DNN-Based Embeddings

Rather than manually encoding users/items, modern systems learn embeddings jointly with the recommendation model.

Example using PyTorch:

import torch
import torch.nn as nn

class DeepRecSys(nn.Module):
    def __init__(self, num_users, num_items, embed_dim=64):
        super().__init__()
        self.user_embedding = nn.Embedding(num_users, embed_dim)
        self.item_embedding = nn.Embedding(num_items, embed_dim)
        self.fc = nn.Sequential(
            nn.Linear(embed_dim * 2, 128),
            nn.ReLU(),
            nn.Linear(128, 1),
            nn.Sigmoid()
        )

    def forward(self, user_ids, item_ids):
        u = self.user_embedding(user_ids)
        i = self.item_embedding(item_ids)
        x = torch.cat([u, i], dim=-1)
        return self.fc(x)

---

Contrastive Learning for Better Embeddings

Contrastive learning (like in SimCLR, MoCo, CLIP) can create more robust embeddings:

• Positive pair: (user, liked content)
• Negative pair: (user, ignored content)

The goal is to pull positive pairs together in embedding space and push negatives apart.

TikTok and Spotify both explore these to learn embeddings without explicit labels.

---

Privacy and Ethical Concerns

User embeddings can infer sensitive attributes (e.g., gender, sexual orientation, political bias), even if not explicitly given.

Risks:

• Manipulative targeting
• Echo chambers
• Algorithmic bias
• Privacy leaks (reverse-engineering from vectors)

Best Practices:

• Add differential privacy noise
• Use adversarial training to debias embeddings
• Limit vector dimensionality and regularize aggressively

---

Multi-Tower Architectures

Popular in real-world systems like YouTube:

• Separate towers for user and item features (e.g., demographics, context, session)
• Joint training with shared loss
• Optimized for approximate nearest neighbor (ANN) retrieval

# Pseudocode: YouTube DNN-style twin tower
UserTower:  [UserID + Behavior history] --> Dense --> User Embedding
ItemTower:  [ItemID + Metadata] --> Dense --> Item Embedding
Dot(User, Item) --> Click prediction

---

Final Thoughts

User embeddings aren’t just a mathematical construct — they’re a mirror to who we are. But with great power comes responsibility. As engineers, we must:

• Guard privacy
• Prevent misuse
• Optimize for long-term wellbeing, not just short-term clicks

In the next part of this series, we’ll explore retrieval mechanisms, approximate search, and how companies build real-time embedding retrieval engines at scale.

---