> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lunarmc.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Concept Clusters

> Named clusters over the analysis-text embedding space — auto-generated weekly via k-means + GPT-4o-mini

Concept Clusters group a tenant's creatives into a handful of named, summarized concepts based on cosine similarity over the analysis-text embedding. Each cluster has a human-readable name (e.g. "UGC founder testimonial — vertical, fast-paced"), a one-sentence summary, and 30-day spend / ROAS aggregates.

<Info>
  Replaces the earlier `creative_concept_clusters` Bob tool that returned anonymous integer cluster IDs. Materialized weekly so cluster names stay stable across calls in the same week.
</Info>

## How it works

```
Weekly cron: 'creative-refresh-concepts'   (Monday 11:00 UTC)
Task:        marketing_resources.refresh_creative_concepts
Queue:       automations_low

Per tenant with ≥ 6 analysis_text embeddings:
   1. Pull last-90d analysis_text embeddings   →  (creative_id, vector)
   2. Pick k = max(2, min(8, n_rows / 40 + 2))
   3. KMeans(n_clusters=k, random_state=0)     →  labels + centroids
   4. For each cluster with ≥ 3 members:
        a. Aggregate tag distribution across members (top-3 per category)
        b. Sample 3 analysis prose summaries
        c. Ask GPT-4o-mini for { name, summary }
        d. Compute spend_30d, roas_30d from _perf_for_window
        e. Persist a CreativeConcept row
   5. Wholesale replace the tenant's prior CreativeConcept rows
```

Re-running on the same tenant deletes the prior rows first — names refresh as the underlying creative library evolves. Clusters that no longer have ≥ 3 members are dropped.

## Naming pass

For each cluster the task asks `gpt-4o-mini` (cheap, fast):

```
Name a cluster of marketing creatives in 4-8 words and write a 1-sentence
summary of what unites them. Return STRICT JSON:
  { "name": "...", "summary": "..." }

Tag distribution by category: { "format": {"ugc": 5, "studio_shot": 1}, ... }
Sample summaries: ["A founder demonstrating ...", "Customer testimonial ...", ...]
```

The prompt is short on purpose — the LLM gets enough signal from tag counts + 3 sample summaries to produce a meaningful name. Cost is about \$0.003 per cluster.

If the naming call fails for any reason, the cluster is still persisted with `name="Cluster N"` and `summary=""`. The page still renders it — you just lose the readable label until the next refresh.

## What's in a cluster

| Field                 | Type           | Meaning                                                                        |
| --------------------- | -------------- | ------------------------------------------------------------------------------ |
| `concept_id`          | UUID           | Primary key.                                                                   |
| `client_id`           | string         | Tenant.                                                                        |
| `name`                | string         | Human-readable name (≤ 160 chars).                                             |
| `summary`             | text           | 1–2 sentences.                                                                 |
| `member_creative_ids` | JSON list      | All `creative_id`s in the cluster.                                             |
| `member_count`        | int            | Length of `member_creative_ids`.                                               |
| `centroid`            | `vector(1536)` | k-means centroid (used for "find this concept's nearest neighbors" if needed). |
| `spend_30d`           | float          | Sum of member creatives' last-30d spend.                                       |
| `roas_30d`            | float          | Average member ROAS, last 30 days.                                             |
| `model`               | string         | Naming-pass model (`gpt-4o-mini`).                                             |
| `generated_at`        | datetime       | When this row was written.                                                     |
| `status`              | string         | `ready` or `stale` (set when last refresh found too few embeddings).           |

## Endpoints

<ParamField path="GET /api/marketing_resources/creatives/concept-clusters/" query>
  Read-only list of materialized concepts for the tenant.

  **Query params:**

  * `client_id` — required (or implicit for single-tenant users).

  Returns `{ "client_id": "...", "concepts": [...] }` ordered by `(spend_30d desc, roas_30d desc)`.
</ParamField>

<ParamField path="POST /api/marketing_resources/creatives/concept-clusters/refresh/" body>
  Force a recluster outside the weekly schedule. Useful after a sweep of new creatives lands.

  Body: `{ "client_id": "..." }`. Returns `202 Accepted` with a Celery task id. The page polls for completion and reloads.
</ParamField>

Both endpoints are tenant-scoped via `shared.auth.get_accessible_client_ids`.

## Bob tool

```python theme={null}
@tool
def creative_concept_clusters(config, client_id: str = '') -> str:
    """List the named concept clusters in the user's creative library."""
```

Reads materialized rows — never recomputes. If no clusters exist for the tenant yet (e.g. first run before the Monday refresh), the tool returns:

```json theme={null}
{
  "client_id": "...",
  "concepts": [],
  "reason": "concepts not yet materialized; POST /api/marketing_resources/creatives/concept-clusters/refresh/ to build them"
}
```

Bob is configured to chain that hint into a friendly suggestion to the user: "I don't see any concept clusters yet — let me kick off a refresh."

## Bubble chart UI

The Ad Creative Analytics page renders concepts as a Highcharts bubble chart:

| Axis / encoding | Maps to                                                    |
| --------------- | ---------------------------------------------------------- |
| X axis          | `spend_30d` (USD)                                          |
| Y axis          | `roas_30d`                                                 |
| Bubble size     | `member_count`                                             |
| Bubble color    | Static brand purple (`#a855f7`)                            |
| Tooltip         | `name`, `summary`, `spend_30d`, `roas_30d`, `member_count` |

A grid of cards below the chart shows each cluster with its name, summary, and stats. Clicking a bubble (or a card — wired in v2) opens a drawer with the member creative thumbnails so the user can quickly scan the actual content of a cluster.

## Worked example

After a tenant's Monday refresh, the page might show:

| Cluster                                  | Members | Spend (30d) | ROAS (30d) |
| ---------------------------------------- | ------- | ----------- | ---------- |
| Bright UGC product reveal — vertical     | 12      | \$4,820     | 2.6        |
| Founder talking heads — slow-burn pacing | 7       | \$2,100     | 3.1        |
| Studio hero shot with bold typography    | 9       | \$3,200     | 1.4        |
| Before / after split-screen              | 5       | \$1,450     | 0.9        |

The Studio hero and Before / after clusters are clearly underperforming relative to the others — that's a useful prompt for the daily brief to recommend leaning into the founder talking heads concept next.

## Cost & cadence

* Embedding the corpus is a one-time cost (\~\$0.0002 per creative; shared with similarity / search).
* k-means is local CPU compute (sklearn).
* Naming pass is the only ongoing cost: \~$0.003 per cluster × ~5 clusters × weekly = ~$0.07 per tenant per month.

For a 100-tenant prod, weekly concept refresh costs about **\$7 / month**.

## Tunable knobs

| Constant (in `refresh_creative_concepts.py`) | Default                           | Purpose                                                                     |
| -------------------------------------------- | --------------------------------- | --------------------------------------------------------------------------- |
| `CONCEPT_NAMING_MODEL_DEFAULT`               | `gpt-4o-mini`                     | Name + summary model. Override via Django settings: `CONCEPT_NAMING_MODEL`. |
| `CLUSTER_SIZE_FLOOR`                         | 3                                 | Drop clusters with fewer members.                                           |
| `since_days` (lookback)                      | 90                                | Embeddings older than this are excluded.                                    |
| k auto-pick                                  | `max(2, min(8, n_rows / 40 + 2))` | Force a fixed k via the `--k` arg on the management command.                |

## Where the code lives

* `marketing_resources/tasks/refresh_creative_concepts.py` — the cron task + naming pass.
* `marketing_resources/models/creative_concept.py` — the row.
* `marketing_resources/views/creative_concepts.py` — the two endpoints.
* `ai_chat/tools_creative.py:creative_concept_clusters` — the Bob tool.