# TEDUH Register

**Nav:** Portal → Analyze Property → **TEDUH Register** pill (`/manage/property/catalog/teduh`), with five
in-page views: *Search register*, *Match review*, *Bulk-buy opportunities*, *Insights*, *Area analysis*
(`TeduhNav.vue`).

## What it does

TEDUH (`teduh.kpkt.gov.my`, the housing ministry's Jabatan Perumahan Negara) publishes every licensed
private housing project in Malaysia: developer, phase, status and % progress, CCC / VP dates, and **every
unit with its price and whether it is sold**. The owner asked (2026-09-26) for:

1. **An archive of the whole register** (~24,900 projects), units included.
2. **A search page like the ministry's own** — search type (Projek / Pemaju), keyword, state → district →
   town, project status, minimum / maximum price.
3. **An AI match of each register project to our published new projects** (the 65 on
   `/manage/property/catalog/new-projects?tab=published`), which an admin confirms, corrects or rejects.
4. **An AI match of each UNSOLD unit to one of our layouts.** The register gives price only: no size and no
   bedrooms. The register shows SPA prices, and ours may be SPA or net of a ~10% rebate. The owner said
   "dun use logic but use gemini", so Gemini does this.
5. **The bulk-buy read**: all of a project's unsold units bought at once, with market value, rent,
   cashflow and ROI, "so that we can do bulk purchase for the remaining units if the roi is good".

## How it works

### Pipeline (runs project by project)

| Step | Command | Writes |
|---|---|---|
| Crawl the list (20 a page, ~20 s a page, **DIRECT** — ScraperAPI returns 500 on it) | `teduh:crawl --phase=list` | `storage/app/teduh/list/page-NNNNN.json` |
| Crawl detail + units (ScraperAPI plain mode, 1 credit a request, ~4 s) | `teduh:crawl --phase=detail --concurrency=20 [--limit=N]` | `storage/app/teduh/projects/{code}.json` |
| Import, match, units | `teduh:process --watch --shard=k/n` | `teduh_projects`, `teduh_project_matches`, `teduh_units`, `teduh_regions` |
| Independent accuracy check | `teduh:audit --phase=import\|match\|units --sample=30` | `storage/app/teduh/audits/{phase}-{time}.json` |

- **Resumable.** Each file that exists is never fetched again. `failures.json` lists what failed.
- **The source throttles with HTTP 429.** Two list pages out of three failed in one burst. The list
  crawl now re-queues a throttled page up to 5 times and pauses a minute. The region lookups retry with
  a 15 s-step backoff. Without it a district's towns silently go missing: the first region sync stored
  48 districts, the real count is 106.
- **Detail crawls linked projects first.** They are what the unit matching and the bulk read wait on.
- **`teduh:process` shards** split matching by `id % n`, one Gemini call at a time per shard (~15 s each).
  Shard 0 also imports new list pages as they land. `--watch` exits once no crawl is running and nothing
  is left.
- **Kill / restart the shards from a SCRIPT FILE, never an inline `pkill -f`.** The pattern matches
  the tool's own `bash -c` command line and kills your shell (exit 144).

### Project matching (`TeduhPipeline::match`, `TeduhMatcher::project`)

- `TeduhTargets::candidatesFor()` offers our published projects **within 2 km**. It also offers any
  within 15 km that share a distinctive name word (5+ letters, a stoplist of township words such as
  jaya / indah / bukit). "Jaya" once paired Taman Karangan Jaya with The Atera @ Petaling Jaya.
- **No candidates → `NO_CANDIDATES`, and no AI call.** That is ~90% of the register.
- Otherwise Gemini (`gemini-3.1-pro-preview`; there is no `gemini-3.1-pro`, it 404s) picks one uuid or
  none. The answer is rejected unless the uuid is one of the candidates.
- `saveMatch` never overwrites a CONFIRMED / CORRECTED / REJECTED row, so an admin's call survives a
  re-run. `LINKED = [AI_MATCHED, CONFIRMED, CORRECTED]`: an unconfirmed AI match already drives the unit
  matching, and the opportunities page flags it "match not yet confirmed".
- **Changing a match to another project deletes that project's units** (their layouts were the old
  project's), and the pipeline re-imports and re-matches them.

### Unit → layout matching (`TeduhMatcher::units`)

- Layouts go to Gemini as keys `L1..Ln`, each with name, bedrooms, sqft, units count, our price and dual-key.
- Every unit in the project goes along as context, as `no|price|sold/unsold`. Sold units show the stack
  pattern. Only unsold units are assigned, 80 per call, with a `timeout` of 240. The 60 s default timed
  out with nothing returned.
- The prompt (`resources/prompts/teduh_unit_layout.md`) sets the rules:
  - ±10% of our price after floor premiums.
  - A stack shares a layout.
  - No layout gets more units than it has.
  - **More than ~20% from every layout → null.** Added after the audit caught a unit 40% above a layout
    and still assigned to it.
  - Reasons name the layout, never the `L` key.
- `saveUnitLayouts` skips units an admin confirmed or corrected, and `replaceUnits` keeps those rows
  when the register is refreshed.

### Bulk-buy arithmetic (`TeduhOpportunities`)

It computes nothing the Layout Analysis did not. Each unit's **market value** (`fair_value`), **rent**
(`rent_median`) and **maintenance** come from its layout's `layout_analyses` row, the same figures the
New Project page shows. `layout_analyses` is on the MASTER connection: it is read in its own query and
never joined. From the register come the asking price (`selling_price`, else `spa_price`) less the
**bulk discount**, and the loan on that.

```
bulk      = asking × (1 − discount)
equity    = market value − bulk
cashflow  = rent − instalment(bulk × loan%, rate, tenure) − maintenance      (monthly)
gross     = rent × 12 / bulk            net = (rent − maintenance) × 12 / bulk
cash ROI  = cashflow × 12 / (bulk − loan)       — null at a 100% loan
```

Defaults: 10% discount; the Layout Analysis loan of 100% at 4% over 35 years, so the numbers line up with
that tab. Every figure is a sum over units, so a layout row and a project row use the same code. The
terms reload the page (`?discount=&loan=&rate=&tenure=`), so there is one implementation of the maths and
no JS twin (see the Layout Analysis rent-parity history).

### Validation (the owner's gate: "do test first and validate using AI LLM, once ok only proceed")

`teduh:audit` has **Claude (`claude-sonnet-5`)** re-judge a random sample of Gemini's work. It is a
different model from the one that did the work.

| Phase | Result | Report |
|---|---|---|
| Project match | 94.1% (16 correct / 1 wrong / 1 unsure) — the "wrong" one's own note says it is right | `audits/match-*.json` |
| Unit layouts | 93.3% (28 / 2) — one real miss (a unit 40% above a layout), which led to the prompt's hard limit | `audits/units-20260927-042753.json` |

Tests: `tests/Feature/Teduh/TeduhPipelineTest.php` covers parsing, the coordinate swap, reviews surviving
re-runs, and unit resets. `TeduhPagesTest.php` covers the search filters, the show page's units, the
review actions and their validation, and the bulk arithmetic.

### Project page and the two calculated SPA terms (2026-09-27)

The project page (`Show.vue` + `Partials/RegisterDetail.vue`) shows what the register holds, with TEDUH's
own Malay labels:
- **Status Terkini Projek**: every component row.
- **Maklumat Perjanjian Jual Beli.**
- **The developer and licence.**
- **The permit.**
- **The advertisement PDFs**: linked from HIMS, not downloaded. About 30% of projects file one, ~0.5 MB each.
- **The register's GPS on a keyless Google Maps embed.** 62% of projects have usable coordinates:
  9,462 file none, and 50 are nonsense and dropped.

Two SPA lines are OURS, not the register's (owner's definitions), and are marked *Calculated*:

- **Final Construction Period** = the higher of Tempoh Pembinaan Asal and Tempoh Pembinaan Baharu
  (`TeduhProject::finalConstructionMonths`).
- **Estimated Handover Date** = Tarikh PJB Pertama + Final Construction Period, month-end safe
  (`TeduhProject::estimatedHandoverDate`). There is none when the project has no first SPA yet (~9,600).

Both are stored as `teduh_projects.final_construction_months` / `estimated_handover_date`. They are written
on import by `TeduhPayload::fromDetail`, so the list can sort by them. Blanks sort last in either
direction.

**Size and PSF ranges** (`TeduhProject::sizeAndPsf`, stored as `size_min_sqft` / `size_max_sqft` /
`psf_min` / `psf_max`, shown under the price) come from each component row's Keluasan Binaan. 14,574
projects have them; most high-rise components file 0. Three traps are handled:

- **The field says m² (Mps), but many developers file sq ft.** A 900 "m²" terrace at RM 282k would be
  RM 29 psf. Each figure is read as m² unless that is implausible (over 600 m², or a psf outside
  RM 60–5,000 while sq ft is plausible).
- **The register pairs no price with a size.** A row's range pairs its cheapest price with its smallest
  size and its dearest price with its largest size. Each end is kept only when plausible on its own.
- **One size with a >1.6× price spread gives no top-end psf.** That spread is penthouses the single
  figure does not describe. Before this rule, 506 projects read over RM 2,000 psf and the average top
  was RM 30,472. After it, 69 read over RM 2,000 psf (some real — Setia V, KLCC — some typos in the
  register itself) and the average top is RM 437.

### Inventory mix and Insights (2026-09-27)

**Inventory mix** (`teduh_projects.mix`, from `TeduhPayload::mix`) records, for each component, its type,
bedrooms, size and price ranges, units, sold and % sold. The list's *Units sold — by type* column shows it.
There, blocks of the same type and bedroom count are merged, the four largest are shown, and the column
is sortable by overall sold %. A *Sales* filter covers has-unsold units and sell-through (<30% / 30–70% /
≥70%).

How the mix is assembled:
- **The register's sold flag lives on UNITS**, which come in unit groups.
- **Size and bedrooms live on the Status Terkini component ROWS.**
- **`TeduhPayload::components` pairs each unit group with a row by position.** When the unit list is
  shorter than the rows (~1 in 10 projects), a group takes the first unused row of the same type instead.

**Insights** (`/teduh/insights`, `TeduhInsightsController` → `Services/TeduhInsights`) shows sold against
unsold by price, size, psf, bedrooms, type and state, plus a price × size heatmap.

The data behind it:
- **`teduh_unit_facts` has one row per unit**, about 2.26M.
- **A unit's size is its component's** when that is one figure. Across a range it is placed by price.
- **`teduh_unit_cubes` pre-aggregates the facts** (about 34k rows) in one INSERT … SELECT, so a page takes
  about 0.5 s. A scan of the facts took about 6 s per 118k rows.
- **Rebuild with `teduh:facts`**: `--shard=k/n` runs in parallel, `--missing` resumes and `--cube-only`
  rebuilds the cube. Parallel shards can deadlock on the facts index, so the transaction retries 5 times.
- **Imports write each project's facts** (`applyDetail`), but **the cube is only rebuilt by
  `teduh:facts`**. The Insights cache key carries the cube's max id, so a rebuild invalidates it.

Traps on the Insights side:
- **The default scope is projects launched (first SPA) in the last 5 years.** Across all years about 80%
  of units are sold (decades-old completed projects), which says nothing about what sells now.
- **The heatmap is shaded relative to the view** (weakest → strongest cell). On absolute 20% steps, every
  cell was the darkest shade.
- **A bedroom count of 0 is "not filed", not studio.** 1.2M units averaging RM 939k carried it.

### Area analysis (2026-09-27)

`/teduh/area` (`TeduhAreaController`, `Area.vue` + `Partials/AreaMap.vue`) shows every placed project as a
dot, all the same size (owner's call, 2026-09-27).

**Colour** is its % sold on a red → green gradient. The bands are `TeduhArea::SOLD_BANDS`: under 30%,
30–50%, 50–70%, 70–90%, 90–99% and 100% sold (key `sold_out`, never `'100'`: PHP turns a numeric string
key into an int, and a strict test on it matched 0 of ~9.6k projects). `soldBins.js` colours them.

**The legend is a filter.** Tick bands (`sold[]`) and both the dots and the area analysis keep only
projects in them. `TeduhArea::bandOf` is the PHP twin of the JS `bandOf`.

**Property type** is a checkbox list (`Partials/TypePicker.vue`, `types[]`). None ticked means all
types. Ticking types changes the dots too: each dot then counts only those types' units, summed from
the project's `mix` components that have a unit list. Summing `teduh_unit_facts` took 12–16 s; the mix
takes 0.2 s. Projects with none of the ticked types drop off the map.

**The area analysis reads exactly the dots on screen** inside the shape. The bands are judged on each
dot's own figures.

**Hover and click:** hovering near a dot shows its name, % sold, sold/units, unsold, price range and
launch year. Clicking opens the project in a new tab; in Draw mode a click adds a point.

**Trap: everything is on ONE canvas renderer**, the map's default, with `tolerance: 6`. With a separate
canvas for the dots, the area circle's canvas stacked on top and swallowed every mouse event: no
tooltip, no click.

The reader defines an area in one of two ways:
- **Radius**: click a centre, default 2 km, slider 0.5–10 km.
- **Draw**: click points, then Finish.

The same breakdown as Insights (`Partials/InsightsBody.vue`, shared by both pages) is computed for just
the projects inside, plus those projects, most unsold first.

- **The area lives in the URL**, so it can be shared: `?mode=radius&lat&lng&radius_km`, or
  `?mode=polygon&poly=lat,lng;…`.
- **Changing the area reloads only the `area` prop**, which is a closure, so a partial reload skips the
  ~5.6k dots.
- **`TeduhArea` finds the projects inside**: a bounding box plus `ST_Distance_Sphere` for a radius, and a
  bounding box plus PHP ray casting for a polygon. Only projects with the register's GPS (~62%) can be
  placed.
- **The cube has no location, so an area is analysed from `teduh_unit_facts`** of its project ids
  (`TeduhInsights::readProjects`, about 1 s for a 2 km KL radius). The whole market still reads the cube.
- **The tiles are OpenStreetMap's.** CARTO's `basemaps.cartocdn.com/light_all` began answering every tile
  with "API KEY REQUIRED" on 2026-09-27. The other maps that were on CARTO (PoiMap, ComparisonMap,
  NewSupplyPanel, AmenitiesPanel, and the two dark ones — ProjectInvestmentLeafletMap and the Analyze
  Property LocationPicker) moved to OpenStreetMap on 2026-10-01 through the shared
  [`baseTileLayer()`](/resources/js/utils/leafletTiles.js); this map still names the URL itself.

The list filters codes as STRINGS (`applyInString`), and the request overrides `newFilteredQuery`: the
Diver base drops any value PHP calls empty, so `?status=0` (Belum Mula) used to filter nothing.

Beyond the register's own form (owner, 2026-10-01): **State is a row of checkboxes** (`state[]`, several
at once; none ticked = every state). Only KL, Selangor, Penang and Johor are offered
(`TeduhProjectQueryRequest::STATE_CHOICES`, applied by the controller after the cached region list). The
District dropdown lists the districts of every ticked state. **The whole form is folded behind a Filters
button on every visit**; the button shows how many filters are applied.
**Property type** (`types[]`, the shared `TypePicker`) keeps a project when ANY component in its `mix` is a
ticked type (`JSON_CONTAINS`); the options are `TeduhProject::mixTypes()`, most projects first, cached 1 h
as `teduh:mix-types`. **Unsold units — at least** (`unsold_min`) keeps projects with `units_unsold >= N`. **Est. handover
(completion)** is one select: *Within 1 year* / *Within 2 years* (`handover_within=1|2`,
`TeduhProjectQueryRequest::HANDOVER_WITHIN` — today to today + N years, so a saved link stays relative and
a date already passed is out) or *Custom dates…*, which shows from / to (`handover_from`, `handover_to`).
All filter the stored `estimated_handover_date`; a project with no first SPA date has no estimate, so any
handover filter drops it.

## Public link (owner, 2026-10-01)

The register search, plus a read-only project page, behind a **secret link with no login**:
`/teduh/{token}` and `/teduh/{token}/{uuid}` (`Main\Teduh\RegisterController`, `routes/main.php`,
`throttle:120,1`). An admin with `manage-projects` makes, replaces or switches off the link from the
**Public link** button on the search page. The token is `Setting::TEDUH_PUBLIC_TOKEN`. Any other
token gets a 404, so a replaced link stops working at once.

- **It shows only what the register publishes.** There is no "Ours" badge and no AI match, layouts,
  review buttons, summary or other tabs. The public request (`Main\Teduh\RegisterQueryRequest`)
  also drops the `matched` filter, which would otherwise let a stranger list every project we sell.
- **Not indexed:** the page sends `X-Robots-Tag: noindex, nofollow` and a robots meta tag.
  `Referrer-Policy: same-origin` keeps the token out of the Referer sent to Google Maps.
- **One codebase for both views.** `Src\Teduh\Services\TeduhRegisterView` builds the props, and
  `$names = null` means the public view. The Vue side is `Partials/RegisterSearch.vue`,
  `ProjectOverview.vue` and `UnitsTable.vue`; the Manage page adds its layout column through
  UnitsTable's `layout` slot. `teduh` is a reserved funnel slug.

## Traps

- **The register's coordinates are sometimes swapped** (lat 101, lng 3). `TeduhPayload::fromListRow`
  puts them back and keeps only Malaysian ranges.
- **`phase_name` is often generic** ("PHASE 2", "FASA 3A", "NIL", digits). `displayName()` falls back to
  or prefixes the project name.
- **A price of RM 270,000 in an otherwise ~RM 800k project is usually affordable-housing quota** (e.g.
  Amara Residences). "No layout fits" is the right answer there.
- **wk reads the catalogue read-only.** Everything here is a SITE table (`teduh_*`, migration in
  `database/migrations/`), keyed to catalogue rows by uuid.

## Related files

- `app/Console/Commands/{CrawlTeduh,ProcessTeduh,AuditTeduh}.php`
- `src/Teduh/` — `TeduhProject`, `TeduhProjectMatch`, `TeduhUnit`, `TeduhRegion`, `TeduhPayload`,
  `Repositories/TeduhProjectRepository`, `Services/{TeduhTargets,TeduhMatcher,TeduhPipeline,TeduhOpportunities}`
- `app/Http/Controllers/Manage/Property/{TeduhController,TeduhOpportunitiesController}.php`
- `app/Http/Requests/Manage/Property/{TeduhProjectQueryRequest,ReviewTeduhMatchRequest,ReviewTeduhUnitRequest,TeduhOpportunitiesRequest}.php`
- `resources/js/Pages/Manage/Property/Teduh/{Index,Show,Matches,Opportunities}.vue` + `Partials/`
- `resources/prompts/teduh_{project_match,unit_layout,match_audit}.md` (registered in `AiRequest` + `config/ai_prompts.php`)
- `database/migrations/2026_09_26_140000_create_teduh_tables.php`