rajch_ales/product-import-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add batch picture candidate workflow

Browse Source
This commit is contained in:
rajch_ales
2026-08-06 00:16:25 +02:00
parent a8e3e62dbd
commit 0987edd112
5 changed files with 289 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
agent.md
+21
View File
@@ -167,8 +167,29 @@ There are three selectable data sources:
- Picture candidates must show source, image dimensions and the proposed target assignment. The target assignment is not always color: supported strategies are `color`, `size`, `combination` and `manual`. Start by using manual corrections from the user, then learn reusable rules in Local scrape DB.
- Scraped pictures and their metadata belong in Local scrape DB, not browser localStorage. Store the binary image in `scrape_image_blobs` with `sha256`, `perceptual_hash`, mime type, dimensions and size. Store each source/target proposal in `scrape_picture_candidates`.
- Every picture candidate row must have its own save action. Saving one candidate stores only that selected image candidate into Local scrape DB and never saves the whole source result list.
- Picture candidate popups must also provide `Save all` for robot/automatic flows. `Save all` saves every currently visible picture candidate one by one and must not save hidden, stale or unrelated candidates.
- Source candidate popups should provide `Get all pictures` for robot/automatic flows. It fetches picture candidates from every currently visible mapped source, then opens the picture candidate popup. Saving happens only in the picture candidate popup.
- Use `sha256` to remove exact duplicate image files and a perceptual hash to flag visually similar images from different sources or sizes. Exact duplicates may be collapsed automatically; visual duplicates should remain reviewable until the user approves the rule.
- Product source cache is cleared when a new product loads.
### Picture Scraping Agent
- Treat picture scraping as a separate internal role: find sources, fetch picture candidates, classify target assignment, detect duplicates and prepare save actions.
- The picture scraping agent stores all scraper settings, candidates, hashes, metadata and learned target rules in Local scrape DB (`catalog_scrape`).
- Picture files are stored as binary blobs in Local scrape DB with disk-friendly metadata, not in browser localStorage and not in the live catalog database.
- The agent starts with user-guided target assignment (`color`, `size`, `combination`, `manual`) and turns repeated corrections into reusable Local scrape DB rules.
- The agent may compare Hudy, Idealo and future sources, but it must keep all candidates reviewable until an exact duplicate or learned rule is trusted.
- The agent never writes to Live 9bplus DB. Live data may only be read later for comparison when explicitly allowed.
#### Picture Scraping Skills
- Source discovery: use manufacturer mapping from Local scrape DB and show exactly which source produced each candidate.
- Browser scraping: use the configured browser engine per source and keep the UI responsive while scraping runs.
- Image extraction: collect original image URLs, dimensions, mime type, file size and source page URL.
- Duplicate detection: calculate `sha256` for exact duplicates and perceptual hash for visually similar images.
- Target assignment: propose whether pictures belong to `color`, `size`, `combination` or `manual` target, then learn from user corrections.
- Persistence: save picture blobs, candidates, hashes, source metadata and learned rules only into Local scrape DB.
- Review UI: show picture size, source, target assignment, duplicate state, per-row save and batch save actions.
- Hudy picture parsing uses the correct product color variant, checks EANs and extracts large gallery images.
- Idealo uses the browser adapter and is intended mainly as a picture source.
- Image preview is square.