Initial catalog maker baseline

This commit is contained in:
2026-08-05 12:25:23 +02:00
commit 2e7d83b679
30 changed files with 5437 additions and 0 deletions
+16
View File
@@ -0,0 +1,16 @@
node_modules/
.env
.env.*
!.env.example
config/local.json
outputs/
logs/
tmp/
*.log
*.sql
*.sql.gz
# Keep source Excel files local unless we explicitly decide otherwise.
*.xlsm
*.xlsx
*.xls
+38
View File
@@ -0,0 +1,38 @@
# Catalog Maker by Magic AI
Node.js replacement for the Excel macro workflow behind the `Catalog maker - 20` button.
## Current Goal
Replace the Excel click workflow with a controlled Node.js pipeline:
1. Read local configuration from `config/local.json`.
2. Load or identify the source workbook.
3. Run catalog-maker steps in a safe order.
4. Default to dry-run mode.
5. Read from the database only. Database writes are forbidden.
## First Run
```powershell
copy config/local.example.json config/local.json
npm run catalog:dry-run
```
## Safety Rules
- Local secrets stay in `config/local.json`.
- `config/local.json` and Excel files are ignored by git.
- The app starts in dry-run mode.
- Rule #1: the database is read-only for this project.
- SQL guards must reject write statements before execution.
## Excel Entry Point Found
- Sheet: `endpoint`
- Button caption: `Catalog maker - 20`
- Internal control name: `CommandButton20`
- Approximate location: `A21:C22`
- ActiveX relation: `xl/activeX/activeX72.xml`
The exact VBA click handler still needs to be mapped before we implement real catalog generation logic.
+116
View File
@@ -0,0 +1,116 @@
# Catalog Maker Project Notes
## Purpose
This project replaces the Excel/VBA workflow named `Catalog maker - 20` with a Node.js local web application. The UI is a compact product catalog editor for loading manufacturer products, combinations, source mappings and parsed supplier data.
## Non-negotiable rules
- Live DB is never written to. Live DB access is read-only through the existing endpoint.
- Local DB may later allow controlled writes, but writes must remain explicit, logged and protected by transaction/dry-run settings.
- Never print, commit or share endpoint tokens, usernames or passwords.
- Never use Live DB as a fallback when Local DB is selected. If Local DB is unavailable, show an error and keep the UI running.
- Keep the development server on port `3404` unless the user explicitly requests another port.
- Do not delete or revert user files or unrelated changes.
## Git workflow
- After every completed implementation step, run the relevant checks, create a focused commit and push it to the configured remote repository.
- Use short, descriptive commit messages that state the completed change.
- Never commit `.env`, `config/local.json`, passwords, tokens, the Excel workbook or database dumps.
- Before the first push, initialize Git if needed, configure the project remote and create an initial baseline commit from safe files only.
- If a push fails, do not expose credentials in the terminal output or in chat; report the failure without printing secrets.
## Runtime
- Start the web app with `npm run dev`.
- Open `http://127.0.0.1:3404/`.
- The current server is `src/server.js`.
- Configuration is loaded from `config/local.json`, merged over `config/local.example.json`.
- `.env` exists for local MariaDB notes, but the application does not yet load `.env` automatically. Do not assume `.env` is active until configuration loading is explicitly wired to it.
## Database architecture
There are two selectable data sources:
### Live DB
- Uses the existing 9b-plus API endpoint configured in `config/local.json`.
- Credentials are secret and must stay server-side.
- Must remain read-only.
### Local DB
- MariaDB is installed locally as service `MariaDB`.
- Host: `localhost` / `127.0.0.1`.
- Port: `3306`.
- Database: `9bplus`.
- A full dump from `9bplus (1).sql.gz` was imported locally.
- Verified local contents: about 266 tables, 74 manufacturers, 8,888 products and 48,206 product combinations.
- The import was local only and did not contact Live DB.
- The project currently needs to be switched to `database.driver = "mariadb"` and supplied with local credentials before the UI can read this database.
## Database safety behavior
- The UI has `Local DB` and `Live DB` settings.
- The UI sends the selected mode using the `X-Database-Mode` request header.
- Backend data API routes reject a Local DB request when MariaDB is not configured instead of silently using the endpoint.
- Backend data API routes reject a Live DB request when the project is configured only for MariaDB.
- Static HTML/CSS/JS files must remain available even when a database is not configured.
- The top status shows the selected source, connection state, permissions and execution mode.
## Important current files
- `src/server.js`: HTTP server, API routes, mode guard and status endpoint.
- `src/config.js`: configuration loading and database settings.
- `src/endpoint/endpoint-client.js`: endpoint queries and MariaDB routing.
- `src/db/mariadb-client.js`: MariaDB pool, read queries, local writes and connection test.
- `src/services/catalog-products.js`: product, combinations, mapping, source and picture workflow.
- `public/index.html`: compact catalog UI and settings modal.
- `public/app.js`: UI state, API calls, Local/Live mode header, local settings and source workflow.
- `public/styles.css`: compact modern layout.
- `config/local.example.json`: safe configuration template.
- `config/local.json`: local secrets and machine-specific configuration; never expose it.
## Current UI behavior
- Manufacturer and language selects load from the selected database.
- Product data loads when manufacturer changes or navigation arrows are used.
- `Get product info` remains a separate action.
- EAN scan input searches on Enter.
- `Find sources` searches once and caches candidates for the current product; a second click reopens cached results.
- Source candidates show how they were found, including EAN/reference/name and color context.
- Each source row has its own `Get pictures` action.
- Product source cache is cleared when a new product loads.
- Hudy picture parsing uses the correct product color variant, checks EANs and extracts large gallery images.
- Idealo uses the browser adapter and is intended mainly as a picture source.
- Image preview is square.
- Settings is at the bottom of the left panel; database status is directly above it.
## Settings
The settings modal contains:
- Local DB / Live DB selection.
- Local DB connection fields: host, port, database, user and password.
- `Test connection`, which performs a read-only `SELECT VERSION()` against local MariaDB only.
- Local insert/update/delete permission switches.
- Transaction confirmation or dry-run preview mode.
- Live DB permissions are always read-only.
Local UI settings are stored in browser localStorage. They are not yet a replacement for server-side `config/local.json` credentials.
## Next safe step
Wire the local connection settings into server configuration without exposing Live credentials, then set the active local database driver to MariaDB and test manufacturer loading from the imported `9bplus` database. After that, verify that switching to Live uses only read-only endpoint queries.
## Verification commands
```powershell
node --check src/server.js
node --check src/db/mariadb-client.js
node --check public/app.js
npm test
```
For a local MariaDB check, use the installed client against `127.0.0.1:3306` and never the Live endpoint.
+27
View File
@@ -0,0 +1,27 @@
{
"appMode": "dry-run",
"sourceWorkbook": "./xxx_endpoint_bo_2026-08-04_v1.xlsm",
"outputDir": "./outputs",
"server": {
"host": "127.0.0.1",
"port": 3400
},
"endpoint": {
"url": "https://www.9b-plus.com/apiv1/endpoint/query",
"loginUrl": "https://www.9b-plus.com/apiv1/auth/login",
"token": "",
"username": "",
"password": ""
},
"database": {
"driver": "endpoint",
"mode": "local",
"allowWrites": false,
"host": "127.0.0.1",
"port": 3306,
"name": "catalog_maker_test",
"user": "catalog_maker",
"password": "",
"connectionLimit": 5
}
}
+78
View File
@@ -0,0 +1,78 @@
# Excel Map
Source workbook:
`xxx_endpoint_bo_2026-08-04_v1.xlsm`
## Sheets
| Sheet | Role hypothesis |
| --- | --- |
| `dashboard` | Cron/task overview and links |
| `endpoint` | Main working screen, includes `Catalog maker - 20` |
| `pictures` | Product image layout/status |
| `picmagic` | Product variants or image-combination helper |
| `sql` | Stored SQL statements and descriptions |
| `cronjob` | Scheduled task metadata |
| `config` | Local configuration and sensitive values |
## Catalog Maker Button
The entry point is an ActiveX button:
- Caption: `Catalog maker - 20`
- Internal name: `CommandButton20`
- Sheet: `endpoint`
- Approximate anchor: `A21:C22`
- Relationship target: `../activeX/activeX72.xml`
## Migration Notes
- The workbook contains VBA macros.
- It has an external link to an older local `.xlsm`.
- Many defined names look like product attributes but currently resolve to `#REF!`.
- The `sql` sheet appears to hold most data-access behavior.
- We should migrate one behavior at a time and keep all write operations disabled until verified.
## Manufacturer SQL
Excel rows found on the `sql` sheet and confirmed from the VBA strings:
- Row 133: `catalog maker / ComboBox2000 / load all manufacturer`
- Row 134: `catalog maker / ComboBox2000 / get id of selected manufacture`
Original Excel flow:
1. Load manufacturer names into `ComboBox2000`.
2. After selecting a manufacturer, resolve `id_manufacturer`.
3. Store/use the selected id for the next catalog-maker queries.
The web endpoint follows the same connector flow, but improves the select by loading ids with names:
```sql
SELECT id_manufacturer, name
FROM ps_manufacturer
ORDER BY name ASC
```
After a user selects a manufacturer, the next step resolves its id with row 134:
```sql
SELECT id_manufacturer
FROM ps_manufacturer
WHERE name = ?
```
## Languages SQL
Excel row found on the `sql` sheet:
- Row 173: `catalog maker / get available lang for translations`
The web endpoint loads language ids, names, and ISO codes:
```sql
SELECT id_lang, name, iso_code
FROM ps_lang
ORDER BY name ASC
```
+473
View File
@@ -0,0 +1,473 @@
{
"name": "catalog-maker-by-magic-ai",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "catalog-maker-by-magic-ai",
"version": "0.1.0",
"dependencies": {
"geckodriver": "^6.1.1",
"mariadb": "^3.5.3",
"selenium-webdriver": "^4.46.0"
},
"engines": {
"node": ">=20"
}
},
"node_modules/@bazel/runfiles": {
"version": "6.5.0",
"resolved": "https://registry.npmjs.org/@bazel/runfiles/-/runfiles-6.5.0.tgz",
"integrity": "sha512-RzahvqTkfpY2jsDxo8YItPX+/iZ6hbiikw1YhE0bA9EKBR5Og8Pa6FHn9PO9M0zaXRVsr0GFQLKbB/0rzy9SzA==",
"license": "Apache-2.0"
},
"node_modules/@types/geojson": {
"version": "7946.0.16",
"resolved": "https://registry.npmjs.org/@types/geojson/-/geojson-7946.0.16.tgz",
"integrity": "sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg==",
"license": "MIT"
},
"node_modules/@types/node": {
"version": "26.1.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.1.2.tgz",
"integrity": "sha512-Vu4a5UFA9rIIFJ7rB/Vaafh9lrCQszopTCx6KjFboXTGQbPNasehVR5TEiithSDGyd1DEiUByggTZsg8jukeIg==",
"license": "MIT",
"dependencies": {
"undici-types": "~8.3.0"
}
},
"node_modules/@wdio/logger": {
"version": "9.29.1",
"resolved": "https://registry.npmjs.org/@wdio/logger/-/logger-9.29.1.tgz",
"integrity": "sha512-0ZAEIo6PNyMIJPlOGkIgyOJUjcd0pC8/QHlVAAe1c91/IcjZ1X+k0yidXHaboJdN7dq1XPUacmhRdtua0U5EZg==",
"license": "MIT",
"dependencies": {
"chalk": "^5.1.2",
"loglevel": "^1.6.0",
"loglevel-plugin-prefix": "^0.8.4",
"safe-regex2": "^5.0.0",
"strip-ansi": "^7.1.0"
},
"engines": {
"node": ">=18.20.0"
}
},
"node_modules/@zip.js/zip.js": {
"version": "2.8.34",
"resolved": "https://registry.npmjs.org/@zip.js/zip.js/-/zip.js-2.8.34.tgz",
"integrity": "sha512-+6a3lyqq69rpseLbvDPiVIWsZ/HdTGAAD6afFtug6ECPDGttb2dHnPC6cJgdPofYkzL9OvXizegq+DQVfL2rnA==",
"license": "BSD-3-Clause",
"engines": {
"bun": ">=0.7.0",
"deno": ">=1.0.0",
"node": ">=18.0.0"
}
},
"node_modules/agent-base": {
"version": "7.1.4",
"resolved": "https://registry.npmjs.org/agent-base/-/agent-base-7.1.4.tgz",
"integrity": "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==",
"license": "MIT",
"engines": {
"node": ">= 14"
}
},
"node_modules/ansi-regex": {
"version": "6.2.2",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-6.2.2.tgz",
"integrity": "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg==",
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/chalk/ansi-regex?sponsor=1"
}
},
"node_modules/chalk": {
"version": "5.6.2",
"resolved": "https://registry.npmjs.org/chalk/-/chalk-5.6.2.tgz",
"integrity": "sha512-7NzBL0rN6fMUW+f7A6Io4h40qQlG+xGmtMxfbnH/K7TAtt8JQWVQK+6g0UXKMeVJoyV5EkkNsErQ8pVD3bLHbA==",
"license": "MIT",
"engines": {
"node": "^12.17.0 || ^14.13 || >=16.0.0"
},
"funding": {
"url": "https://github.com/chalk/chalk?sponsor=1"
}
},
"node_modules/core-util-is": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.3.tgz",
"integrity": "sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ==",
"license": "MIT"
},
"node_modules/debug": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
"integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/decamelize": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/decamelize/-/decamelize-6.0.1.tgz",
"integrity": "sha512-G7Cqgaelq68XHJNGlZ7lrNQyhZGsFqpwtGFexqUv4IQdjKoSYF7ipZ9UuTJZUSQXFj/XaoBLuEVIVqr8EJngEQ==",
"license": "MIT",
"engines": {
"node": "^12.20.0 || ^14.13.1 || >=16.0.0"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/denque": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
"integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",
"license": "Apache-2.0",
"engines": {
"node": ">=0.10"
}
},
"node_modules/geckodriver": {
"version": "6.1.1",
"resolved": "https://registry.npmjs.org/geckodriver/-/geckodriver-6.1.1.tgz",
"integrity": "sha512-/AcCyc9o9o6hUbudaSJM2iOtXbxSLqQPOb4GrPvEN40cjraUeaX/j5kH3mSgwiroyMn7qzx2wM61AQ2XY8j3sA==",
"hasInstallScript": true,
"license": "MIT",
"dependencies": {
"@wdio/logger": "^9.18.0",
"@zip.js/zip.js": "^2.8.11",
"decamelize": "^6.0.1",
"http-proxy-agent": "^7.0.2",
"https-proxy-agent": "^7.0.6",
"modern-tar": "^0.7.3"
},
"bin": {
"geckodriver": "bin/geckodriver.js"
},
"engines": {
"node": ">=20.0.0"
}
},
"node_modules/http-proxy-agent": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/http-proxy-agent/-/http-proxy-agent-7.0.2.tgz",
"integrity": "sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==",
"license": "MIT",
"dependencies": {
"agent-base": "^7.1.0",
"debug": "^4.3.4"
},
"engines": {
"node": ">= 14"
}
},
"node_modules/https-proxy-agent": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz",
"integrity": "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==",
"license": "MIT",
"dependencies": {
"agent-base": "^7.1.2",
"debug": "4"
},
"engines": {
"node": ">= 14"
}
},
"node_modules/iconv-lite": {
"version": "0.7.3",
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.3.tgz",
"integrity": "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ==",
"license": "MIT",
"dependencies": {
"safer-buffer": ">= 2.1.2 < 3.0.0"
},
"engines": {
"node": ">=0.10.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/immediate": {
"version": "3.0.6",
"resolved": "https://registry.npmjs.org/immediate/-/immediate-3.0.6.tgz",
"integrity": "sha512-XXOFtyqDjNDAQxVfYxuF7g9Il/IbWmmlQg2MYKOH8ExIT1qg6xc4zyS3HaEEATgs1btfzxq15ciUiY7gjSXRGQ==",
"license": "MIT"
},
"node_modules/inherits": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC"
},
"node_modules/isarray": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz",
"integrity": "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ==",
"license": "MIT"
},
"node_modules/jszip": {
"version": "3.10.1",
"resolved": "https://registry.npmjs.org/jszip/-/jszip-3.10.1.tgz",
"integrity": "sha512-xXDvecyTpGLrqFrvkrUSoxxfJI5AH7U8zxxtVclpsUtMCq4JQ290LY8AW5c7Ggnr/Y/oK+bQMbqK2qmtk3pN4g==",
"license": "(MIT OR GPL-3.0-or-later)",
"dependencies": {
"lie": "~3.3.0",
"pako": "~1.0.2",
"readable-stream": "~2.3.6",
"setimmediate": "^1.0.5"
}
},
"node_modules/lie": {
"version": "3.3.0",
"resolved": "https://registry.npmjs.org/lie/-/lie-3.3.0.tgz",
"integrity": "sha512-UaiMJzeWRlEujzAuw5LokY1L5ecNQYZKfmyZ9L7wDHb/p5etKaxXhohBcrw0EYby+G/NA52vRSN4N39dxHAIwQ==",
"license": "MIT",
"dependencies": {
"immediate": "~3.0.5"
}
},
"node_modules/loglevel": {
"version": "1.9.2",
"resolved": "https://registry.npmjs.org/loglevel/-/loglevel-1.9.2.tgz",
"integrity": "sha512-HgMmCqIJSAKqo68l0rS2AanEWfkxaZ5wNiEFb5ggm08lDs9Xl2KxBlX3PTcaD2chBM1gXAYf491/M2Rv8Jwayg==",
"license": "MIT",
"engines": {
"node": ">= 0.6.0"
},
"funding": {
"type": "tidelift",
"url": "https://tidelift.com/funding/github/npm/loglevel"
}
},
"node_modules/loglevel-plugin-prefix": {
"version": "0.8.4",
"resolved": "https://registry.npmjs.org/loglevel-plugin-prefix/-/loglevel-plugin-prefix-0.8.4.tgz",
"integrity": "sha512-WpG9CcFAOjz/FtNht+QJeGpvVl/cdR6P0z6OcXSkr8wFJOsV2GRj2j10JLfjuA4aYkcKCNIEqRGCyTife9R8/g==",
"license": "MIT"
},
"node_modules/lru-cache": {
"version": "11.5.2",
"resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.5.2.tgz",
"integrity": "sha512-4pfM1Ff0x50o0tQwb5ucw/RzNyD0/YJME6IVcStalZuMWxdt3sR3huStTtxz4PUmvZfRguvDejasvQ2kifR11g==",
"license": "BlueOak-1.0.0",
"engines": {
"node": "20 || >=22"
}
},
"node_modules/mariadb": {
"version": "3.5.3",
"resolved": "https://registry.npmjs.org/mariadb/-/mariadb-3.5.3.tgz",
"integrity": "sha512-i053Kc0MgdUv/hu9mCyq67TYfPXFj3/MV8I7ZW5wvJNixIyXC0VztMPUjIVj/449nQo+BsxFD4Fdk/sA/uqKPQ==",
"license": "LGPL-2.1-or-later",
"dependencies": {
"@types/geojson": "^7946.0.16",
"@types/node": ">=20",
"denque": "^2.1.0",
"iconv-lite": "^0.7.2",
"lru-cache": "^11.5.0"
},
"engines": {
"node": ">= 20.0.0"
}
},
"node_modules/modern-tar": {
"version": "0.7.7",
"resolved": "https://registry.npmjs.org/modern-tar/-/modern-tar-0.7.7.tgz",
"integrity": "sha512-t9VmxaqrmANnEOBhpSDI6HD192Ge48k8vmWqQQL7hSFEqHEYwZbbsu49+aKLWZeRvFs3j1pMhXOqqF4kPlvjkQ==",
"license": "MIT",
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT"
},
"node_modules/pako": {
"version": "1.0.11",
"resolved": "https://registry.npmjs.org/pako/-/pako-1.0.11.tgz",
"integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==",
"license": "(MIT AND Zlib)"
},
"node_modules/process-nextick-args": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.1.tgz",
"integrity": "sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==",
"license": "MIT"
},
"node_modules/readable-stream": {
"version": "2.3.8",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-2.3.8.tgz",
"integrity": "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==",
"license": "MIT",
"dependencies": {
"core-util-is": "~1.0.0",
"inherits": "~2.0.3",
"isarray": "~1.0.0",
"process-nextick-args": "~2.0.0",
"safe-buffer": "~5.1.1",
"string_decoder": "~1.1.1",
"util-deprecate": "~1.0.1"
}
},
"node_modules/ret": {
"version": "0.5.0",
"resolved": "https://registry.npmjs.org/ret/-/ret-0.5.0.tgz",
"integrity": "sha512-I1XxrZSQ+oErkRR4jYbAyEEu2I0avBvvMM5JN+6EBprOGRCs63ENqZ3vjavq8fBw2+62G5LF5XelKwuJpcvcxw==",
"license": "MIT",
"engines": {
"node": ">=10"
}
},
"node_modules/safe-buffer": {
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.2.tgz",
"integrity": "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g==",
"license": "MIT"
},
"node_modules/safe-regex2": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/safe-regex2/-/safe-regex2-5.1.1.tgz",
"integrity": "sha512-mOSBvHGDZMuIEZMdOz/aCEYDCv0E7nfcNsIhUF+/P+xC7Hyf3FkvymqgPbg9D1EdSGu+uKbJgy09K/RKKc7kJA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fastify"
},
{
"type": "opencollective",
"url": "https://opencollective.com/fastify"
}
],
"license": "MIT",
"dependencies": {
"ret": "~0.5.0"
},
"bin": {
"safe-regex2": "bin/safe-regex2.js"
}
},
"node_modules/safer-buffer": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz",
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
"license": "MIT"
},
"node_modules/selenium-webdriver": {
"version": "4.46.0",
"resolved": "https://registry.npmjs.org/selenium-webdriver/-/selenium-webdriver-4.46.0.tgz",
"integrity": "sha512-UlTkgnx9y+bf3QxFsrgUyMqC2oy1Yf+qcOs7xIC/XfsUPv/ow7Sicx6SJ7AeOn2/Z+xF1M8SLqyn983wipI82w==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/SeleniumHQ"
},
{
"type": "opencollective",
"url": "https://opencollective.com/selenium"
}
],
"license": "Apache-2.0",
"dependencies": {
"@bazel/runfiles": "^6.5.0",
"jszip": "^3.10.1",
"tmp": "^0.2.7",
"ws": "^8.21.0"
},
"engines": {
"node": ">= 20.0.0"
}
},
"node_modules/setimmediate": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/setimmediate/-/setimmediate-1.0.5.tgz",
"integrity": "sha512-MATJdZp8sLqDl/68LfQmbP8zKPLQNV6BIZoIgrscFDQ+RsvK/BxeDQOgyxKKoh0y/8h3BqVFnCqQ/gd+reiIXA==",
"license": "MIT"
},
"node_modules/string_decoder": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.1.1.tgz",
"integrity": "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==",
"license": "MIT",
"dependencies": {
"safe-buffer": "~5.1.0"
}
},
"node_modules/strip-ansi": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-7.2.0.tgz",
"integrity": "sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w==",
"license": "MIT",
"dependencies": {
"ansi-regex": "^6.2.2"
},
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/chalk/strip-ansi?sponsor=1"
}
},
"node_modules/tmp": {
"version": "0.2.7",
"resolved": "https://registry.npmjs.org/tmp/-/tmp-0.2.7.tgz",
"integrity": "sha512-e0votIpp4Uo2AJYSzVHV6xCcawuiez3DzqDAbrTc3YxBkplN6e+dM13ZeIcZnDg/QpSuU2zfZ3rzwY8ukEnaXw==",
"license": "MIT",
"engines": {
"node": ">=14.14"
}
},
"node_modules/undici-types": {
"version": "8.3.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.3.0.tgz",
"integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==",
"license": "MIT"
},
"node_modules/util-deprecate": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
"license": "MIT"
},
"node_modules/ws": {
"version": "8.21.2",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.21.2.tgz",
"integrity": "sha512-54dMVAo4WIe6SKy3vBgN+9bJZqqQ8IMRevAkOLQALhi49qkkQDQfWdAZ8KQlXiEabw88ARXXdUrlvtbKQX+aKw==",
"license": "MIT",
"engines": {
"node": ">=10.0.0"
},
"peerDependencies": {
"bufferutil": "^4.0.1",
"utf-8-validate": ">=5.0.2"
},
"peerDependenciesMeta": {
"bufferutil": {
"optional": true
},
"utf-8-validate": {
"optional": true
}
}
}
}
}
+21
View File
@@ -0,0 +1,21 @@
{
"name": "catalog-maker-by-magic-ai",
"version": "0.1.0",
"private": true,
"type": "module",
"description": "Node.js replacement for the Excel Catalog maker - 20 workflow.",
"scripts": {
"start": "node src/cli.js",
"dev": "node src/server.js",
"catalog:dry-run": "node src/cli.js catalog-maker --dry-run",
"test": "node --test"
},
"engines": {
"node": ">=20"
},
"dependencies": {
"geckodriver": "^6.1.1",
"mariadb": "^3.5.3",
"selenium-webdriver": "^4.46.0"
}
}
+1151
View File
File diff suppressed because it is too large Load Diff
+190
View File
@@ -0,0 +1,190 @@
<!doctype html>
<html lang="cs">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Catalog Maker</title>
<link rel="stylesheet" href="/styles.css" />
<script src="/app.js" type="module"></script>
</head>
<body>
<main class="catalog-screen">
<aside class="control-panel">
<div class="app-status">
<span class="safe-mode" id="databaseStatus">DB read-only</span>
</div>
<div class="record-id" id="recordId">&nbsp;</div>
<div class="arrow-row">
<button class="arrow-button arrow-button-prev" id="previousProductButton" type="button" title="Previous product" aria-label="Previous product"></button>
<button class="arrow-button arrow-button-next" id="nextProductButton" type="button" title="Next product" aria-label="Next product"></button>
</div>
<div class="parser-actions" aria-label="Parser actions">
<button class="action-button" id="findSourcesButton" type="button">Find sources</button>
<button class="action-button" id="getPicturesButton" type="button">Get pictures</button>
<button class="action-button" type="button">Get features</button>
<button class="action-button" type="button">Get texts</button>
<button class="action-button" type="button">Preview parsed data</button>
<button class="action-button" type="button" disabled>Apply / Save</button>
</div>
<button class="action-button" id="loadProductButton" type="button">Get product info - 00</button>
<button class="action-button" type="button">Creat combin. - 02</button>
<input class="input-like" id="scanInput" aria-label="Scan EAN" placeholder="Scan EAN" />
<div class="picture-slot" id="pictureSlot">
<span>No picture</span>
</div>
<select class="input-like" id="languageSelect" aria-label="Language">
<option value="">Select language</option>
</select>
<select class="input-like" id="manufacturerSelect" aria-label="Manufacturer">
<option value="">Select manufacturer</option>
</select>
<div class="mapping-box" id="mappingBox">
<details class="mapping-details" id="mappingDetails">
<summary>
<span class="mapping-state is-missing" id="mappingState">Mapping not loaded</span>
<span class="mapping-toggle" aria-hidden="true"></span>
</summary>
<div class="mapping-list" id="mappingList"></div>
</details>
</div>
<button class="action-button primary" type="button">Show menu (20)</button>
<button class="action-button" type="button">Scraper - 03</button>
<button class="action-button" type="button">BG remover - 04</button>
<button class="action-button" id="settingsButton" type="button">Settings</button>
</aside>
<section class="product-panel">
<div class="field-grid" id="fieldGrid"></div>
</section>
<section class="data-panel">
<div class="table-wrap">
<table class="variant-table">
<thead>
<tr id="variantHeader"></tr>
</thead>
<tbody>
<tr>
<td class="empty-row" id="emptyTableMessage" colspan="13">Select manufacturer to load data.</td>
</tr>
</tbody>
</table>
</div>
</section>
</main>
<div class="modal-backdrop" id="suggestedModal" hidden>
<section class="suggested-dialog" aria-labelledby="suggestedTitle">
<header class="suggested-header">
<h2 id="suggestedTitle">Sugested product for</h2>
<button class="modal-close" id="suggestedCloseButton" type="button">Close</button>
</header>
<div class="suggested-list" id="suggestedList"></div>
</section>
</div>
<div class="modal-backdrop" id="sourceModal" hidden>
<section class="source-dialog" aria-labelledby="sourceTitle">
<header class="suggested-header">
<h2 id="sourceTitle">Source candidates</h2>
<button class="modal-close" id="sourceCloseButton" type="button">Close</button>
</header>
<div class="source-list" id="sourceList"></div>
</section>
</div>
<div class="modal-backdrop" id="settingsModal" hidden>
<section class="settings-dialog" aria-labelledby="settingsTitle">
<header class="suggested-header">
<h2 id="settingsTitle">Settings</h2>
<button class="modal-close" id="settingsCloseButton" type="button">Close</button>
</header>
<div class="settings-content">
<fieldset class="settings-group">
<legend>Database</legend>
<label class="settings-option">
<input type="radio" name="databaseMode" value="local" checked />
<span>Local DB</span>
</label>
<label class="settings-option">
<input type="radio" name="databaseMode" value="live" />
<span>Live DB</span>
</label>
<div class="local-db-fields" id="localDbFields">
<label class="settings-field">
<span>Host</span>
<input id="localDbHost" type="text" autocomplete="off" value="127.0.0.1" />
</label>
<label class="settings-field">
<span>Port</span>
<input id="localDbPort" type="number" inputmode="numeric" value="3306" />
</label>
<label class="settings-field">
<span>Database</span>
<input id="localDbName" type="text" autocomplete="off" value="catalog_maker_test" />
</label>
<label class="settings-field">
<span>User</span>
<input id="localDbUser" type="text" autocomplete="username" value="root" />
</label>
<label class="settings-field">
<span>Password</span>
<input id="localDbPassword" type="text" autocomplete="off" />
</label>
<button class="settings-test-button" id="testLocalDbButton" type="button">Test connection</button>
<p class="settings-note" id="localDbTestResult">Connection not tested.</p>
</div>
<p class="settings-note" id="databaseModeNote">Local test database selected.</p>
</fieldset>
<fieldset class="settings-group">
<legend>Access permissions</legend>
<label class="settings-option">
<input type="checkbox" id="allowReadToggle" checked disabled />
<span>Allow read</span>
</label>
<label class="settings-option">
<input type="checkbox" id="allowInsertToggle" />
<span>Allow insert</span>
</label>
<label class="settings-option">
<input type="checkbox" id="allowUpdateToggle" />
<span>Allow update</span>
</label>
<label class="settings-option">
<input type="checkbox" id="allowDeleteToggle" />
<span>Allow delete</span>
</label>
<p class="settings-note">Live DB is always read-only. Write permissions apply only to Local DB.</p>
</fieldset>
<fieldset class="settings-group">
<legend>Execution safety</legend>
<label class="settings-option">
<input type="radio" name="transactionMode" value="transaction" checked />
<span>Transaction mode: confirm before commit</span>
</label>
<label class="settings-option">
<input type="radio" name="transactionMode" value="dry-run" />
<span>Dry-run: preview SQL only</span>
</label>
<div class="settings-permissions">
<span>DROP, TRUNCATE and ALTER remain blocked.</span>
<span>Every write will be logged before execution.</span>
</div>
</fieldset>
</div>
</section>
</div>
</body>
</html>
+852
View File
@@ -0,0 +1,852 @@
:root {
color-scheme: light;
--surface: #f4f6f8;
--panel: #22272e;
--panel-soft: #2d333b;
--panel-line: #3e4651;
--detail-label: #e2e6ea;
--detail-value: #eefdf9;
--detail-border: #c8d3d9;
--table-head: #d9dee4;
--table-row: #f5f6f7;
--table-row-alt: #eceff2;
--table-line: #cfd6dd;
--text: #15181c;
--muted: #68717d;
--accent: #7ee34d;
--accent-text: #10220b;
--safe: #0f7b4d;
--danger: #e02424;
--focus: #2f6fed;
}
* {
box-sizing: border-box;
}
body {
margin: 0;
min-height: 100vh;
color: var(--text);
font-family:
Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Arial,
sans-serif;
background: var(--surface);
}
button,
input,
select {
font: inherit;
}
.catalog-screen {
min-height: 100vh;
display: grid;
grid-template-columns: 172px 292px minmax(720px, 1fr);
}
.control-panel {
min-height: 100vh;
max-height: 100vh;
padding: 6px;
background: var(--panel);
display: flex;
flex-direction: column;
gap: 6px;
overflow-y: auto;
}
.control-panel > #settingsButton {
order: 99;
}
.app-status {
order: 98;
margin-top: auto;
display: flex;
flex-wrap: wrap;
gap: 3px;
padding: 5px;
border: 1px solid var(--panel-line);
border-radius: 6px;
background: var(--panel-soft);
color: #f3f6f8;
font-size: 11px;
line-height: 1.2;
}
.record-id {
height: 39px;
display: grid;
place-items: center;
border: 1px solid rgba(126, 227, 77, 0.42);
border-radius: 6px;
background: linear-gradient(180deg, #9aff60, var(--accent));
color: var(--accent-text);
font-size: 26px;
font-weight: 700;
line-height: 1;
font-variant-numeric: tabular-nums;
}
.arrow-row {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 5px;
}
.arrow-button,
.action-button,
.input-like {
width: 100%;
border: 1px solid var(--panel-line);
border-radius: 5px;
background: #f8fafc;
color: #111827;
}
.arrow-button,
.action-button {
cursor: pointer;
}
.arrow-button {
height: 31px;
position: relative;
display: grid;
place-items: center;
}
.arrow-button::before {
content: "";
width: 0;
height: 0;
border-top: 8px solid transparent;
border-bottom: 8px solid transparent;
}
.arrow-button-prev::before {
border-right: 11px solid #111827;
}
.arrow-button-next::before {
border-left: 11px solid #111827;
}
.arrow-button:disabled::before {
opacity: 0.55;
}
.parser-actions {
display: grid;
gap: 4px;
}
.parser-actions .action-button {
min-height: 28px;
font-size: 11px;
}
.action-button {
min-height: 32px;
padding: 0 8px;
font-size: 12px;
display: grid;
place-items: center;
text-align: center;
}
.action-button.primary {
min-height: 34px;
font-size: 14px;
font-weight: 700;
}
.arrow-button:hover:not(:disabled),
.action-button:hover:not(:disabled) {
border-color: #7a8492;
background: #ffffff;
}
.arrow-button:disabled,
.action-button:disabled {
cursor: default;
opacity: 0.45;
}
.input-like {
height: 28px;
padding: 3px 7px;
font-size: 12px;
outline: none;
}
select.input-like {
text-align: center;
text-align-last: center;
}
#scanInput {
background: #e5e8ec;
text-align: center;
}
.input-like:focus {
border-color: var(--focus);
box-shadow: 0 0 0 2px rgba(47, 111, 237, 0.18);
}
.input-like::placeholder {
color: #8b95a1;
}
.picture-slot {
aspect-ratio: 1 / 1;
width: 100%;
min-height: 0;
display: grid;
place-items: center;
border: 1px dashed #6d7784;
border-radius: 5px;
background: var(--panel-soft);
color: #b8c0ca;
font-size: 12px;
overflow: hidden;
}
.picture-slot img {
width: 100%;
height: 100%;
display: block;
object-fit: contain;
}
.mapping-box {
display: grid;
gap: 3px;
font-size: 11px;
}
.mapping-state {
height: 28px;
display: flex;
align-items: center;
min-width: 0;
padding: 3px 7px;
border: 1px solid var(--panel-line);
border-right: 0;
border-radius: 5px 0 0 5px;
background: #f8fafc;
color: #111827;
font-weight: 400;
line-height: 1.15;
}
.mapping-state.is-ready {
background: #f8fafc;
color: #111827;
}
.mapping-state.is-missing {
background: #fff0f0;
color: #b42318;
font-weight: 700;
}
.mapping-list {
display: grid;
gap: 3px;
}
.mapping-details {
min-width: 0;
}
.mapping-details summary {
display: grid;
grid-template-columns: minmax(0, 1fr) 22px;
align-items: center;
cursor: pointer;
list-style: none;
}
.mapping-details summary::-webkit-details-marker {
display: none;
}
.mapping-details summary:hover {
filter: brightness(1.04);
}
.mapping-toggle {
width: 22px;
height: 28px;
display: grid;
place-items: center;
border: 1px solid var(--panel-line);
border-left: 0;
border-radius: 0 5px 5px 0;
background: #f8fafc;
}
.mapping-toggle::before {
content: ">";
color: #111827;
font-size: 11px;
font-weight: 900;
line-height: 1;
}
.mapping-details[open] .mapping-toggle::before {
transform: rotate(90deg);
}
.mapping-source-row {
min-width: 0;
max-width: 100%;
display: grid;
grid-template-columns: minmax(0, 1fr) auto;
align-items: center;
gap: 4px;
padding: 3px 4px;
border: 1px solid #46505c;
border-radius: 5px;
background: #1b2026;
color: #f3f6f8;
}
.mapping-source-row.is-disabled {
opacity: 0.55;
}
.mapping-source-name {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-weight: 800;
}
.mapping-source-status {
flex: 0 0 auto;
padding: 1px 4px;
border-radius: 4px;
font-size: 9px;
font-weight: 400;
}
.mapping-source-status.is-ok {
background: #e8edf3;
color: #303944;
}
.mapping-source-status.is-bad {
background: #ffe0dc;
color: #b42318;
font-weight: 700;
}
.mapping-source-status.is-muted {
background: #4b5563;
color: #e5e7eb;
}
.product-panel {
min-width: 0;
padding: 22px 0 8px;
background: #d3d8de;
border-right: 1px solid #bdc6cf;
}
.field-grid {
display: grid;
grid-template-columns: 132px minmax(0, 168px);
}
.field-label,
.field-value {
min-height: 22px;
padding: 4px 8px;
border-bottom: 1px solid var(--detail-border);
font-size: 12px;
line-height: 1.15;
}
.field-label {
color: #26313d;
background: var(--detail-label);
}
.field-value {
background: var(--detail-value);
text-align: right;
overflow-wrap: anywhere;
font-variant-numeric: tabular-nums;
}
.field-value[data-field="manufacturer_name"] {
color: #b42318;
font-weight: 700;
}
.data-panel {
min-width: 0;
background: var(--surface);
}
.safe-mode,
#modePill {
padding: 2px 6px;
border-radius: 5px;
background: #e8f7ef;
color: var(--safe);
font-weight: 700;
}
#databaseStatus.database-error {
background: #fde8e7;
color: #b42318;
}
#modePill {
background: #edf1f5;
color: #4d5966;
font-weight: 600;
}
.table-wrap {
height: 100vh;
overflow: auto;
}
.variant-table {
width: 100%;
min-width: 820px;
border-collapse: separate;
border-spacing: 0;
table-layout: fixed;
font-size: 12px;
background: var(--table-row);
}
.variant-table th,
.variant-table td {
height: 22px;
border-right: 1px solid var(--table-line);
border-bottom: 1px solid var(--table-line);
padding: 2px 7px;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
font-variant-numeric: tabular-nums;
}
.variant-table th {
position: sticky;
top: 0;
z-index: 2;
background: var(--table-head);
color: #303944;
font-weight: 700;
text-align: center;
}
.variant-table td {
background: var(--table-row);
}
.variant-table tr:nth-child(even) td {
background: var(--table-row-alt);
}
.variant-table tr:hover td {
background: #fff7df;
}
.empty-row {
color: var(--muted);
text-align: center;
font-style: italic;
}
.variant-table .problem-ean {
background: var(--danger) !important;
color: #ffffff;
font-weight: 700;
}
.modal-backdrop {
position: fixed;
inset: 0;
z-index: 20;
display: grid;
place-items: center;
padding: 24px;
background: rgba(17, 24, 39, 0.38);
}
.modal-backdrop[hidden] {
display: none;
}
.suggested-dialog {
width: min(900px, 94vw);
max-height: min(620px, 88vh);
display: grid;
grid-template-rows: auto minmax(0, 1fr);
border: 1px solid #9aa5b1;
border-radius: 8px;
overflow: hidden;
background: #f8fafc;
box-shadow: 0 18px 60px rgba(15, 23, 42, 0.34);
}
.source-dialog {
width: min(820px, 94vw);
max-height: min(620px, 88vh);
display: grid;
grid-template-rows: auto minmax(0, 1fr);
border: 1px solid #9aa5b1;
border-radius: 8px;
overflow: hidden;
background: #f8fafc;
box-shadow: 0 18px 60px rgba(15, 23, 42, 0.34);
}
.settings-dialog {
width: min(520px, 94vw);
max-height: 92vh;
display: grid;
grid-template-rows: auto minmax(0, 1fr);
border: 1px solid #9aa5b1;
border-radius: 8px;
overflow: hidden;
background: #f8fafc;
box-shadow: 0 18px 60px rgba(15, 23, 42, 0.34);
}
.settings-content {
display: grid;
gap: 8px;
padding: 10px;
overflow: visible;
}
.settings-group {
display: grid;
gap: 6px;
margin: 0;
padding: 8px;
border: 1px solid var(--table-line);
border-radius: 6px;
background: #ffffff;
}
.settings-group legend {
padding: 0 5px;
font-size: 12px;
font-weight: 700;
}
.settings-option {
display: flex;
align-items: center;
gap: 8px;
font-size: 12px;
}
.local-db-fields {
display: grid;
gap: 6px;
padding: 8px;
border: 1px solid var(--table-line);
border-radius: 5px;
background: #f7f9fb;
}
.local-db-fields[hidden] {
display: none;
}
.settings-field {
display: grid;
grid-template-columns: 82px minmax(0, 1fr);
align-items: center;
gap: 8px;
font-size: 11px;
}
.settings-field input {
min-width: 0;
padding: 4px 6px;
border: 1px solid #cbd3dc;
border-radius: 4px;
font: inherit;
}
.settings-test-button {
width: 100%;
padding: 5px 8px;
border: 1px solid #b8c3ce;
border-radius: 4px;
background: #eef2f5;
color: #26323d;
cursor: pointer;
}
.settings-test-button:disabled {
opacity: 0.6;
cursor: wait;
}
.connection-ok {
color: #18794e;
}
.connection-error {
color: #b42318;
font-weight: 600;
}
.settings-option input {
margin: 0;
}
.settings-option input:disabled + span {
color: var(--muted);
}
.settings-note,
.settings-permissions {
margin: 0;
color: var(--muted);
font-size: 11px;
line-height: 1.35;
}
.settings-permissions {
display: grid;
gap: 3px;
padding-top: 3px;
}
.suggested-header {
min-height: 42px;
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
padding: 8px 10px;
border-bottom: 1px solid var(--table-line);
background: #e8edf3;
}
.suggested-header h2 {
margin: 0;
font-size: 14px;
}
.modal-close {
border: 1px solid #b6c0ca;
border-radius: 5px;
padding: 5px 10px;
background: #ffffff;
cursor: pointer;
font-size: 12px;
}
.suggested-list {
overflow: auto;
}
.suggested-row {
display: grid;
grid-template-columns: 58px 92px 1fr 160px 64px 84px;
align-items: center;
gap: 8px;
padding: 6px 10px;
border-bottom: 1px solid var(--table-line);
font-size: 12px;
}
.suggested-row:nth-child(even) {
background: #eef2f6;
}
.suggested-row img {
width: 46px;
height: 46px;
object-fit: contain;
border: 1px solid #d3dae2;
background: #ffffff;
}
.suggested-id,
.suggested-active {
text-align: center;
font-variant-numeric: tabular-nums;
}
.suggested-name,
.suggested-reference {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.suggested-pick {
border: 1px solid #2f6fed;
border-radius: 5px;
padding: 5px 8px;
background: #2f6fed;
color: #ffffff;
cursor: pointer;
font-size: 12px;
}
.source-list {
display: grid;
gap: 8px;
padding: 10px;
overflow: auto;
}
.source-section {
display: grid;
gap: 4px;
border: 1px solid var(--table-line);
border-radius: 6px;
overflow: hidden;
background: #ffffff;
}
.source-section-header,
.source-row {
display: grid;
grid-template-columns: minmax(0, 1fr) auto;
align-items: center;
gap: 10px;
min-height: 32px;
padding: 6px 8px;
font-size: 12px;
}
.source-section-header {
background: #e8edf3;
}
.source-section-actions {
display: flex;
align-items: center;
gap: 6px;
}
.source-picture-action {
border: 1px solid #9aa8b7;
border-radius: 4px;
padding: 3px 6px;
background: #ffffff;
color: #172033;
cursor: pointer;
font-size: 10px;
}
.source-picture-action:hover {
background: #e9f2ff;
border-color: #2f6fed;
}
.source-row {
color: #172033;
text-decoration: none;
border-top: 1px solid var(--table-line);
}
.source-row:hover {
background: #fff7df;
}
.source-row-title {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.source-status,
.source-row-action {
padding: 2px 6px;
border-radius: 5px;
font-size: 10px;
font-weight: 800;
}
.picture-candidate {
display: grid;
grid-template-rows: 132px auto;
gap: 6px;
color: inherit;
text-decoration: none;
font-size: 11px;
min-width: 132px;
}
.picture-candidate img {
width: 132px;
height: 132px;
object-fit: contain;
background: #fff;
border: 1px solid #d5dce5;
}
.picture-candidate span {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.source-status.is-ok {
background: #dff7e7;
color: #0f7b4d;
}
.source-status.is-muted,
.source-row-action {
background: #edf1f5;
color: #4d5966;
}
@media (max-width: 980px) {
.catalog-screen {
grid-template-columns: 1fr;
}
.control-panel {
min-height: auto;
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
}
.record-id,
.arrow-row,
.parser-actions,
.app-status,
.mapping-box,
.picture-slot {
grid-column: span 2;
}
.product-panel {
padding: 0;
}
.field-grid {
grid-template-columns: 160px minmax(0, 1fr);
}
.table-wrap {
height: auto;
max-height: 70vh;
}
}
+45
View File
@@ -0,0 +1,45 @@
import fs from "node:fs";
import { createRequire } from "node:module";
const require = createRequire(import.meta.url);
const JSZip = require("jszip");
const workbookPath = process.argv[2] || "xxx_endpoint_bo_2026-08-04_v1.xlsm";
const fromRow = Number.parseInt(process.argv[3] || "1", 10);
const toRow = Number.parseInt(process.argv[4] || String(fromRow), 10);
const zip = await JSZip.loadAsync(fs.readFileSync(workbookPath));
const sharedStringsXml = await zip.file("xl/sharedStrings.xml").async("string");
const sharedStrings = [...sharedStringsXml.matchAll(/<si[\s\S]*?<\/si>/g)].map((match) =>
decodeXml([...match[0].matchAll(/<t[^>]*>([\s\S]*?)<\/t>/g)].map((part) => part[1]).join("")),
);
const sqlSheetXml = await zip.file("xl/worksheets/sheet5.xml").async("string");
for (const rowMatch of sqlSheetXml.matchAll(/<row[^>]* r="(\d+)"[\s\S]*?<\/row>/g)) {
const rowNumber = Number(rowMatch[1]);
if (rowNumber < fromRow || rowNumber > toRow) continue;
const cells = {};
for (const cellMatch of rowMatch[0].matchAll(
/<c[^>]* r="([A-Z]+)\d+"([^>]*)>([\s\S]*?)<\/c>/g,
)) {
const [, column, attributes, body] = cellMatch;
const inlineString = body.match(/<is>[\s\S]*?<t[^>]*>([\s\S]*?)<\/t>[\s\S]*?<\/is>/);
const rawValue = body.match(/<v>([\s\S]*?)<\/v>/)?.[1] ?? inlineString?.[1] ?? "";
cells[column] = attributes.includes('t="s"')
? sharedStrings[Number(rawValue)] || ""
: decodeXml(rawValue);
}
console.log(JSON.stringify({ row: rowNumber, cells }, null, 2));
}
function decodeXml(value) {
return String(value ?? "")
.replaceAll("&lt;", "<")
.replaceAll("&gt;", ">")
.replaceAll("&amp;", "&")
.replaceAll("&quot;", '"')
.replaceAll("&apos;", "'");
}
Binary file not shown.
+24
View File
@@ -0,0 +1,24 @@
#!/usr/bin/env node
import { runCatalogMaker } from "./workflows/catalog-maker.js";
import { loadConfig } from "./config.js";
import { parseArgs } from "./lib/args.js";
import { fail } from "./lib/log.js";
const args = parseArgs(process.argv.slice(2));
const command = args._[0] ?? "catalog-maker";
try {
const config = loadConfig({
configPath: args.config ?? "config/local.json",
dryRun: args["dry-run"] ?? undefined,
});
if (command === "catalog-maker") {
await runCatalogMaker({ config });
} else {
throw new Error(`Unknown command: ${command}`);
}
} catch (error) {
fail(error);
process.exitCode = 1;
}
+67
View File
@@ -0,0 +1,67 @@
import fs from "node:fs";
import path from "node:path";
export function loadConfig({ configPath = "config/local.json", dryRun } = {}) {
const localConfig = readJsonFile(configPath);
const exampleConfig = readJsonFile("config/local.example.json");
const config = mergeConfig(exampleConfig, localConfig);
const appMode = String(config.appMode || "dry-run").toLowerCase();
const resolvedDryRun = dryRun === undefined ? appMode !== "live" : Boolean(dryRun);
return {
appMode,
dryRun: resolvedDryRun,
sourceWorkbook: path.resolve(config.sourceWorkbook || "./xxx_endpoint_bo_2026-08-04_v1.xlsm"),
outputDir: path.resolve(config.outputDir || "./outputs"),
server: {
host: config.server?.host || "127.0.0.1",
port: Number(config.server?.port || 3400),
},
endpoint: {
url: config.endpoint?.url || "https://www.9b-plus.com/apiv1/endpoint/query",
loginUrl: config.endpoint?.loginUrl || "https://www.9b-plus.com/apiv1/auth/login",
token: config.endpoint?.token || "",
username: config.endpoint?.username || "",
password: config.endpoint?.password || "",
},
database: {
driver: String(config.database?.driver || "endpoint").toLowerCase(),
mode: String(config.database?.mode || "local").toLowerCase(),
allowWrites: Boolean(config.database?.allowWrites),
host: config.database?.host || "127.0.0.1",
port: Number(config.database?.port || 3306),
name: config.database?.name || "catalog_maker_test",
user: config.database?.user || "catalog_maker",
password: config.database?.password || "",
connectionLimit: Number(config.database?.connectionLimit || 5),
},
};
}
function readJsonFile(filePath) {
if (!fs.existsSync(filePath)) {
return {};
}
return JSON.parse(fs.readFileSync(filePath, "utf8"));
}
function mergeConfig(base, override) {
return {
...base,
...override,
server: {
...base.server,
...override.server,
},
endpoint: {
...base.endpoint,
...override.endpoint,
},
database: {
...base.database,
...override.database,
},
};
}
+77
View File
@@ -0,0 +1,77 @@
import mariadb from "mariadb";
import { assertReadOnlySql } from "../lib/sql-readonly.js";
let pool;
export function isMariaDbConfigured(config) {
return config.database?.driver === "mariadb";
}
export async function testMariaDbConnection(database) {
const connection = await mariadb.createConnection({
host: database.host,
port: Number(database.port),
database: database.name,
user: database.user,
password: database.password,
bigIntAsNumber: true,
});
try {
const rows = await connection.query("SELECT VERSION() AS version");
return { connected: true, version: rows[0]?.version || "MariaDB" };
} finally {
await connection.end();
}
}
export async function queryMariaDb(config, sql) {
assertReadOnlySql(sql);
const connection = await getPool(config).getConnection();
try {
const rows = await connection.query(sql);
return { configured: true, items: normalizeRows(rows) };
} finally {
connection.release();
}
}
export async function executeMariaDbWrite(config, sql) {
if (config.database.mode !== "local") {
throw new Error("MariaDB writes are blocked outside Local DB mode.");
}
if (!config.database.allowWrites) {
throw new Error("MariaDB writes are disabled. Enable local database writes in settings first.");
}
const connection = await getPool(config).getConnection();
try {
const result = await connection.query(sql);
return {
configured: true,
items: [],
affectedRows: result.affectedRows ?? 0,
insertId: result.insertId ?? 0,
};
} finally {
connection.release();
}
}
async function getPool(config) {
if (!pool) {
pool = mariadb.createPool({
host: config.database.host,
port: config.database.port,
database: config.database.name,
user: config.database.user,
password: config.database.password,
connectionLimit: config.database.connectionLimit,
bigIntAsNumber: true,
});
}
return pool;
}
function normalizeRows(rows) {
return Array.isArray(rows) ? rows.filter((row) => row && typeof row === "object" && !("meta" in row)) : [];
}
+152
View File
@@ -0,0 +1,152 @@
import { assertReadOnlySql } from "../lib/sql-readonly.js";
import { assertAllowedWriteSql } from "../lib/sql-write-allowlist.js";
import { executeMariaDbWrite, isMariaDbConfigured, queryMariaDb } from "../db/mariadb-client.js";
export function isEndpointConfigured(config) {
return Boolean(
config.endpoint.url &&
(config.endpoint.token || (config.endpoint.loginUrl && config.endpoint.username && config.endpoint.password)),
);
}
export async function queryEndpoint(config, sql) {
assertReadOnlySql(sql);
if (isMariaDbConfigured(config)) {
return queryMariaDb(config, sql);
}
if (!isEndpointConfigured(config)) {
return {
configured: false,
items: [],
message: "Endpoint token is not configured in config/local.json.",
};
}
let token = config.endpoint.token;
let response = await sendQuery(config, sql, token);
if (response.status === 401 && config.endpoint.username && config.endpoint.password) {
token = await loginEndpoint(config);
response = await sendQuery(config, sql, token);
}
const text = await response.text();
let data;
try {
data = text ? JSON.parse(text) : {};
} catch {
data = { raw: text };
}
if (!response.ok) {
const message = data?.message || data?.error || response.statusText;
throw new Error(`Endpoint request failed (${response.status}): ${message}`);
}
return normalizeEndpointResponse(data);
}
export async function executeEndpointWrite(config, sql, { operation } = {}) {
if (isMariaDbConfigured(config)) {
return executeMariaDbWrite(config, sql);
}
assertAllowedWriteSql(sql, { operation });
if (!isEndpointConfigured(config)) {
return {
configured: false,
items: [],
message: "Endpoint token is not configured in config/local.json.",
};
}
let token = config.endpoint.token;
let response = await sendQuery(config, sql, token);
if (response.status === 401 && config.endpoint.username && config.endpoint.password) {
token = await loginEndpoint(config);
response = await sendQuery(config, sql, token);
}
const text = await response.text();
let data;
try {
data = text ? JSON.parse(text) : {};
} catch {
data = { raw: text };
}
if (!response.ok) {
const message = data?.message || data?.error || response.statusText;
throw new Error(`Endpoint write failed (${response.status}): ${message}`);
}
return normalizeEndpointResponse(data);
}
async function sendQuery(config, sql, token) {
const form = new FormData();
form.append("query", sql);
return fetch(config.endpoint.url, {
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
Accept: "application/json",
},
body: form,
});
}
async function loginEndpoint(config) {
const form = new FormData();
form.append("username", config.endpoint.username);
form.append("password", config.endpoint.password);
const response = await fetch(config.endpoint.loginUrl, {
method: "POST",
body: form,
});
const text = await response.text();
let data;
try {
data = text ? JSON.parse(text) : {};
} catch {
data = { raw: text };
}
if (!response.ok) {
const message = data?.message || data?.error || response.statusText;
throw new Error(`Endpoint login failed (${response.status}): ${message}`);
}
const token = data?.token || data?.access_token || data?.data?.token || data?.data?.access_token;
if (!token) {
throw new Error("Endpoint login did not return a token.");
}
return token;
}
function normalizeEndpointResponse(data) {
if (Array.isArray(data)) {
return { configured: true, items: data };
}
if (Array.isArray(data.items)) {
return { configured: true, items: data.items, meta: data };
}
if (Array.isArray(data.data)) {
return { configured: true, items: data.data, meta: data };
}
if (Array.isArray(data.rows)) {
return { configured: true, items: data.rows, meta: data };
}
return { configured: true, items: [], meta: data };
}
+36
View File
@@ -0,0 +1,36 @@
export function parseArgs(argv) {
const args = { _: [] };
for (let index = 0; index < argv.length; index += 1) {
const arg = argv[index];
if (!arg.startsWith("--")) {
args._.push(arg);
continue;
}
const [rawKey, inlineValue] = arg.slice(2).split("=", 2);
const key = rawKey.trim();
if (inlineValue !== undefined) {
args[key] = coerceValue(inlineValue);
continue;
}
const next = argv[index + 1];
if (next && !next.startsWith("--")) {
args[key] = coerceValue(next);
index += 1;
} else {
args[key] = true;
}
}
return args;
}
function coerceValue(value) {
if (value === "true") return true;
if (value === "false") return false;
return value;
}
+12
View File
@@ -0,0 +1,12 @@
export function info(message) {
console.log(message);
}
export function warn(message) {
console.warn(message);
}
export function fail(error) {
const message = error instanceof Error ? error.message : String(error);
console.error(`Error: ${message}`);
}
+21
View File
@@ -0,0 +1,21 @@
const writeStatementPattern =
/^(?:alter|analyze|call|create|delete|drop|grant|insert|load|lock|optimize|replace|revoke|set|truncate|update)\b/i;
export function assertReadOnlySql(sql) {
const normalized = stripLeadingSqlNoise(sql);
if (!normalized) {
throw new Error("SQL query is empty.");
}
if (writeStatementPattern.test(normalized)) {
throw new Error(`Blocked non-read-only SQL statement: ${normalized.split(/\s+/, 1)[0].toUpperCase()}`);
}
}
function stripLeadingSqlNoise(sql) {
return String(sql)
.replace(/^\s+/g, "")
.replace(/^(?:--[^\n]*\n|\/\*[\s\S]*?\*\/\s*)+/g, "")
.trim();
}
+27
View File
@@ -0,0 +1,27 @@
const allowedOperation = "catalog-maker-refresh-manufacturer";
export function assertAllowedWriteSql(sql, { operation } = {}) {
if (operation !== allowedOperation) {
throw new Error(`Write SQL blocked. Unsupported operation: ${operation || "none"}`);
}
const compact = compactSql(sql);
const allowed = [
/^updateps_product_catalogpcsetpc\.skladovka=0,pc\.catalog_9b=0wherepc\.id_manufacturer=\d+$/,
/^updateps_product_catalogpcjoinps_import_suppliers_dumpisdonisd\.ean=pc\.ean_13setpc\.skladovka=1wherepc\.id_manufacturer=\d+$/,
/^updateps_product_catalogpcsetpc\.catalog_9b=1wherepc\.id_manufacturer=\d+andpc\.ean_13in\(selectpa\.ean13fromps_product_attributepajoinps_productponp\.id_product=pa\.id_productwherep\.id_manufacturer=\d+\)$/,
];
if (!allowed.some((pattern) => pattern.test(compact))) {
throw new Error("Write SQL blocked. Statement is not on the catalog-maker allowlist.");
}
}
function compactSql(sql) {
return String(sql)
.replace(/`/g, "")
.replace(/\s+/g, " ")
.trim()
.toLowerCase()
.replace(/\s/g, "");
}
+242
View File
@@ -0,0 +1,242 @@
import http from "node:http";
import fs from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { loadConfig } from "./config.js";
import { testMariaDbConnection } from "./db/mariadb-client.js";
import { listManufacturers } from "./services/manufacturers.js";
import { listLanguages } from "./services/languages.js";
import {
loadManufacturerCatalogState,
findProductSources,
listMappingSources,
listSuggestedProducts,
loadProductByEan,
loadProductInfo,
getProductPictures,
} from "./services/catalog-products.js";
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const publicDir = path.resolve(__dirname, "../public");
const config = loadConfig();
const mimeTypes = new Map([
[".html", "text/html; charset=utf-8"],
[".css", "text/css; charset=utf-8"],
[".js", "text/javascript; charset=utf-8"],
[".json", "application/json; charset=utf-8"],
[".svg", "image/svg+xml"],
]);
const server = http.createServer(async (request, response) => {
try {
const url = new URL(request.url, `http://${request.headers.host}`);
if (url.pathname === "/api/status") {
await sendJson(response, getStatus());
return;
}
if (url.pathname === "/api/database/test" && request.method === "POST") {
const body = await readJsonBody(request);
const database = {
driver: "mariadb",
host: body.host || "127.0.0.1",
port: Number(body.port || 3306),
name: body.name || "",
user: body.user || "",
password: body.password || "",
};
const result = await testMariaDbConnection(database);
await sendJson(response, result);
return;
}
const requestConfig = url.pathname.startsWith("/api/")
? getRequestConfig(request)
: null;
if (url.pathname === "/api/manufacturers") {
await sendJson(response, await listManufacturers(requestConfig));
return;
}
if (url.pathname === "/api/languages") {
await sendJson(response, await listLanguages(requestConfig));
return;
}
if (url.pathname === "/api/catalog-maker/product") {
await sendJson(
response,
await loadManufacturerCatalogState(requestConfig, {
manufacturerId: url.searchParams.get("manufacturerId"),
position: url.searchParams.get("position") || 1,
}),
);
return;
}
if (url.pathname === "/api/catalog-maker/mapping-sources") {
await sendJson(
response,
await listMappingSources(requestConfig, {
manufacturerId: url.searchParams.get("manufacturerId"),
checkUrls: url.searchParams.get("check") === "1",
}),
);
return;
}
if (url.pathname === "/api/catalog-maker/find-sources") {
await sendJson(
response,
await findProductSources(requestConfig, {
manufacturerId: url.searchParams.get("manufacturerId"),
sourceKey: url.searchParams.get("sourceKey"),
productName: url.searchParams.get("productName"),
supplierReference: url.searchParams.get("supplierReference"),
ean: url.searchParams.get("ean"),
eanGroup: url.searchParams.get("eanGroup"),
color: url.searchParams.get("color"),
}),
);
return;
}
if (url.pathname === "/api/catalog-maker/get-pictures") {
await sendJson(
response,
await getProductPictures(requestConfig, {
manufacturerId: url.searchParams.get("manufacturerId"),
sourceKey: url.searchParams.get("sourceKey"),
productName: url.searchParams.get("productName"),
supplierReference: url.searchParams.get("supplierReference"),
ean: url.searchParams.get("ean"),
eanGroup: url.searchParams.get("eanGroup"),
color: url.searchParams.get("color"),
}),
);
return;
}
if (url.pathname === "/api/catalog-maker/product-info") {
await sendJson(
response,
await loadProductInfo(requestConfig, {
productId: url.searchParams.get("productId"),
manufacturerId: url.searchParams.get("manufacturerId"),
supplierReference: url.searchParams.get("supplierReference"),
}),
);
return;
}
if (url.pathname === "/api/catalog-maker/product-by-ean") {
await sendJson(
response,
await loadProductByEan(requestConfig, {
manufacturerId: url.searchParams.get("manufacturerId"),
ean: url.searchParams.get("ean"),
}),
);
return;
}
if (url.pathname === "/api/catalog-maker/suggested-products") {
await sendJson(
response,
await listSuggestedProducts(requestConfig, {
manufacturerId: url.searchParams.get("manufacturerId"),
supplierReference: url.searchParams.get("supplierReference"),
productName: url.searchParams.get("productName"),
}),
);
return;
}
const requestedPath = url.pathname === "/" ? "/index.html" : url.pathname;
const safePath = path.normalize(requestedPath).replace(/^(\.\.[/\\])+/, "");
const filePath = path.join(publicDir, safePath);
if (!filePath.startsWith(publicDir)) {
response.writeHead(403);
response.end("Forbidden");
return;
}
const content = await fs.readFile(filePath);
response.writeHead(200, {
"content-type": mimeTypes.get(path.extname(filePath)) || "application/octet-stream",
});
response.end(content);
} catch (error) {
if (error.code === "ENOENT") {
response.writeHead(404);
response.end("Not found");
return;
}
await sendJson(response, {
message: error.message || "Internal server error",
}, 500);
}
});
server.listen(config.server.port, config.server.host, () => {
console.log(`Catalog Maker preview: http://${config.server.host}:${config.server.port}`);
});
function getStatus() {
return {
mode: config.dryRun ? "dry-run" : "live",
databaseAccess: "read-only",
database: {
driver: config.database.driver,
mode: config.database.mode,
allowWrites: config.database.driver === "mariadb" && config.database.mode === "local" && config.database.allowWrites,
},
workbook: path.basename(config.sourceWorkbook),
workbookExists: true,
steps: [
["load-context", "Ready", "Workbook and local config"],
["load-manufacturer", "Mapping", "Manufacturer and product selection"],
["check-stock", "Mapping", "Supplier stock and existing catalog state"],
["build-combinations", "Mapping", "Product combinations"],
["update-prices", "Mapping", "Price calculation preview"],
["update-texts", "Mapping", "Product card text generation"],
["handle-pictures", "Mapping", "Cover and detail pictures"],
],
};
}
function getRequestConfig(request) {
const selectedMode = String(request.headers["x-database-mode"] || config.database.mode).toLowerCase();
if (selectedMode === "local" && config.database.driver !== "mariadb") {
throw new Error("Local DB is selected, but MariaDB is not configured. Live DB was not used.");
}
if (selectedMode === "live" && config.database.driver === "mariadb") {
throw new Error("Live DB is selected, but this project is configured for local MariaDB.");
}
return {
...config,
database: {
...config.database,
mode: selectedMode,
},
};
}
async function sendJson(response, data, statusCode = 200) {
response.writeHead(statusCode, { "content-type": "application/json; charset=utf-8" });
response.end(JSON.stringify(data));
}
async function readJsonBody(request) {
let body = "";
for await (const chunk of request) body += chunk;
return body ? JSON.parse(body) : {};
}
File diff suppressed because it is too large Load Diff
+82
View File
@@ -0,0 +1,82 @@
import { spawn } from "node:child_process";
import { Builder } from "selenium-webdriver";
import firefox from "selenium-webdriver/firefox.js";
import { start } from "geckodriver";
const FIREFOX_PATHS = [
"C:\\Program Files\\Mozilla Firefox\\firefox.exe",
"C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe",
];
export function openInFirefox(url) {
const firefoxPath = FIREFOX_PATHS.find((candidate) => candidate);
if (!firefoxPath) throw new Error("Firefox executable was not found.");
const process = spawn(firefoxPath, [url], {
detached: true,
stdio: "ignore",
windowsHide: true,
});
process.unref();
}
export async function searchInFirefox({ url, headless = false, waitAfterLoadMs = 3000 }) {
const firefoxPath = FIREFOX_PATHS.find((candidate) => candidate);
const port = 4444;
const geckodriver = await start({ port, binary: firefoxPath, log: "error" });
let driver;
try {
await waitForWebDriver(port);
const options = new firefox.Options();
if (firefoxPath) options.setBinary(firefoxPath);
if (headless) options.addArguments("-headless");
driver = await new Builder()
.forBrowser("firefox")
.setFirefoxOptions(options)
.usingServer(`http://127.0.0.1:${port}`)
.build();
await driver.get(url);
await driver.sleep(Math.max(1000, Number(waitAfterLoadMs) || 3000));
const [currentUrl, title, pageSource] = await Promise.all([
driver.getCurrentUrl(),
driver.getTitle(),
driver.getPageSource(),
]);
const cleanText = stripHtml(pageSource).slice(0, 1200);
const failed = /something has gone wrong|reference id/i.test(cleanText);
return { ok: !failed, url: currentUrl, title, text: cleanText, pageSource };
} finally {
if (driver) await driver.quit().catch(() => {});
geckodriver.kill();
}
}
async function waitForWebDriver(port) {
const deadline = Date.now() + 10000;
while (Date.now() < deadline) {
try {
const response = await fetch(`http://127.0.0.1:${port}/status`);
if (response.ok) return;
} catch {
// Geckodriver needs a short moment to open its local port.
}
await new Promise((resolve) => setTimeout(resolve, 150));
}
throw new Error("Firefox WebDriver did not start.");
}
function stripHtml(value) {
return String(value ?? "")
.replace(/<script[\s\S]*?<\/script>/gi, " ")
.replace(/<style[\s\S]*?<\/style>/gi, " ")
.replace(/<[^>]+>/g, " ")
.replace(/&nbsp;/gi, " ")
.replace(/&amp;/gi, "&")
.replace(/\s+/g, " ")
.trim();
}
+14
View File
@@ -0,0 +1,14 @@
import { queryEndpoint } from "../endpoint/endpoint-client.js";
import { catalogMakerSql } from "../sql/catalog-maker.js";
export async function listLanguages(config) {
const result = await queryEndpoint(config, catalogMakerSql.languages);
return {
...result,
items: result.items.map((row) => ({
id: row.id_lang,
name: row.name,
isoCode: row.iso_code,
})),
};
}
+13
View File
@@ -0,0 +1,13 @@
import { queryEndpoint } from "../endpoint/endpoint-client.js";
import { catalogMakerSql } from "../sql/catalog-maker.js";
export async function listManufacturers(config) {
const result = await queryEndpoint(config, catalogMakerSql.manufacturers);
return {
...result,
items: result.items.map((row) => ({
id: row.id_manufacturer,
name: row.name,
})),
};
}
+442
View File
@@ -0,0 +1,442 @@
export const catalogMakerSql = {
manufacturers: `
SELECT id_manufacturer, name
FROM ps_manufacturer
ORDER BY name ASC
`,
manufacturerByName: `
SELECT id_manufacturer
FROM ps_manufacturer
WHERE name = ?
`,
languages: `
SELECT id_lang, name, iso_code
FROM ps_lang
ORDER BY name ASC
`,
};
export function prepareManufacturerCatalogSqls({ manufacturerId }) {
const safeManufacturerId = parsePositiveInteger(manufacturerId, "manufacturerId");
return [
`
UPDATE ps_product_catalog pc
SET pc.skladovka = 0,
pc.catalog_9b = 0
WHERE pc.id_manufacturer = ${safeManufacturerId}
`,
`
UPDATE ps_product_catalog pc
JOIN ps_import_suppliers_dump isd ON isd.ean = pc.ean_13
SET pc.skladovka = 1
WHERE pc.id_manufacturer = ${safeManufacturerId}
`,
`
UPDATE ps_product_catalog pc
SET pc.catalog_9b = 1
WHERE pc.id_manufacturer = ${safeManufacturerId}
AND pc.ean_13 IN (
SELECT pa.ean13
FROM ps_product_attribute pa
JOIN ps_product p ON p.id_product = pa.id_product
WHERE p.id_manufacturer = ${safeManufacturerId}
)
`,
];
}
export function productToDoByManufacturerSql({ manufacturerId, position = 1 }) {
const safeManufacturerId = parsePositiveInteger(manufacturerId, "manufacturerId");
const safePosition = Number.parseInt(position, 10) || 1;
return `
WITH todo_refs AS (
SELECT
supplier_reference,
ROW_NUMBER() OVER (ORDER BY MIN(id_product_catalog) ASC) AS position,
COUNT(*) OVER () AS total
FROM ps_product_catalog
WHERE id_manufacturer = ${safeManufacturerId}
AND catalog_9b = 0
AND skladovka = 1
GROUP BY supplier_reference
),
target_ref AS (
SELECT supplier_reference, position, total
FROM todo_refs
WHERE position = ${safePosition}
),
combination AS (
SELECT pc.*
FROM ps_product_catalog pc
JOIN target_ref tr ON tr.supplier_reference = pc.supplier_reference
WHERE pc.id_manufacturer = ${safeManufacturerId}
AND pc.catalog_9b = 0
AND pc.skladovka = 1
ORDER BY pc.id_product_catalog ASC
LIMIT 1
),
existing_product AS (
SELECT pc.supplier_reference, pc.ean_13
FROM ps_product_catalog pc
JOIN target_ref tr ON tr.supplier_reference = pc.supplier_reference
WHERE pc.id_manufacturer = ${safeManufacturerId}
AND pc.catalog_9b = 1
AND (pc.skladovka = 1 OR pc.skladovka = 0)
ORDER BY pc.id_product_catalog ASC
)
SELECT
co.*,
tr.position,
tr.total,
p.id_product,
p.price,
p.id_supplier,
(
SELECT CONCAT('https://www.9b-plus.com/', i.id_image, '-home_default/', pl.link_rewrite, '.jpg')
FROM ps_image i
JOIN ps_product_lang pl
ON pl.id_product = i.id_product
AND pl.id_lang = 2
WHERE i.id_product = p.id_product
ORDER BY i.cover DESC, i.position ASC
LIMIT 1
) AS image_url
FROM combination co
JOIN target_ref tr ON tr.supplier_reference = co.supplier_reference
LEFT JOIN existing_product ep ON ep.supplier_reference = co.supplier_reference
LEFT JOIN ps_product_attribute pa ON pa.ean13 = ep.ean_13
LEFT JOIN ps_product p ON p.id_product = pa.id_product
LIMIT 1
`;
}
export function productByEanSql({ manufacturerId, ean }) {
const safeManufacturerId = parsePositiveInteger(manufacturerId, "manufacturerId");
const safeEan = escapeSqlString(normalizeEan(ean));
if (!safeEan) {
throw new Error("ean is required.");
}
return `
WITH todo_refs AS (
SELECT
supplier_reference,
ROW_NUMBER() OVER (ORDER BY MIN(id_product_catalog) ASC) AS position,
COUNT(*) OVER () AS total
FROM ps_product_catalog
WHERE id_manufacturer = ${safeManufacturerId}
AND catalog_9b = 0
AND skladovka = 1
GROUP BY supplier_reference
),
combination AS (
SELECT pc.*
FROM ps_product_catalog pc
WHERE pc.id_manufacturer = ${safeManufacturerId}
AND pc.ean_13 = '${safeEan}'
ORDER BY pc.id_product_catalog ASC
LIMIT 1
),
existing_product AS (
SELECT pc.supplier_reference, pc.ean_13
FROM ps_product_catalog pc
JOIN combination co ON co.supplier_reference = pc.supplier_reference
WHERE pc.id_manufacturer = ${safeManufacturerId}
AND pc.catalog_9b = 1
ORDER BY pc.id_product_catalog ASC
)
SELECT
co.*,
tr.position,
tr.total,
p.id_product,
p.price,
p.id_supplier,
(
SELECT CONCAT('https://www.9b-plus.com/', i.id_image, '-home_default/', pl.link_rewrite, '.jpg')
FROM ps_image i
JOIN ps_product_lang pl
ON pl.id_product = i.id_product
AND pl.id_lang = 2
WHERE i.id_product = p.id_product
ORDER BY i.cover DESC, i.position ASC
LIMIT 1
) AS image_url
FROM combination co
LEFT JOIN todo_refs tr ON tr.supplier_reference = co.supplier_reference
LEFT JOIN existing_product ep ON ep.supplier_reference = co.supplier_reference
LEFT JOIN ps_product_attribute pa ON pa.ean13 = ep.ean_13
LEFT JOIN ps_product p ON p.id_product = pa.id_product
LIMIT 1
`;
}
export function productCombinationsByReferenceSql({ manufacturerId, supplierReference }) {
const safeManufacturerId = parsePositiveInteger(manufacturerId, "manufacturerId");
const safeSupplierReference = escapeSqlString(supplierReference);
if (!safeSupplierReference) {
throw new Error("supplierReference is required.");
}
return `
SELECT pc.*
FROM ps_product_catalog pc
WHERE pc.id_manufacturer = ${safeManufacturerId}
AND pc.supplier_reference = '${safeSupplierReference}'
ORDER BY pc.id_product_catalog ASC
`;
}
export function productCombinationInfoSql({ productId, manufacturerId, supplierReference }) {
const safeProductId = parsePositiveInteger(productId, "productId");
const safeManufacturerId = parsePositiveInteger(manufacturerId, "manufacturerId");
const safeSupplierReference = escapeSqlString(supplierReference);
if (!safeSupplierReference) {
throw new Error("supplierReference is required.");
}
return `
WITH catalog_rows AS (
SELECT pc.*
FROM ps_product_catalog pc
WHERE pc.id_manufacturer = ${safeManufacturerId}
AND pc.supplier_reference = '${safeSupplierReference}'
),
attribute_group AS (
SELECT a.id_attribute_group
FROM ps_product_attribute pa
JOIN ps_product_attribute_combination pac
ON pac.id_product_attribute = pa.id_product_attribute
JOIN ps_attribute a
ON a.id_attribute = pac.id_attribute
JOIN ps_attribute_group_lang agl
ON agl.id_attribute_group = a.id_attribute_group
WHERE a.id_attribute_group <> 1
AND agl.id_lang = 1
AND pa.id_product = ${safeProductId}
GROUP BY a.id_attribute_group
ORDER BY a.id_attribute_group ASC
LIMIT 1
),
matched_attributes AS (
SELECT
cr.id_product_catalog,
(
SELECT a.id_attribute
FROM ps_attribute_lang al
JOIN ps_attribute a
ON a.id_attribute = al.id_attribute
WHERE al.name = TRIM(SUBSTRING_INDEX(cr.color, '/', 1))
AND al.id_lang = 1
AND a.id_attribute_group = 1
ORDER BY a.id_attribute ASC
LIMIT 1
) AS color_attribute_id,
(
SELECT al.name
FROM ps_attribute_lang al
JOIN ps_attribute a
ON a.id_attribute = al.id_attribute
WHERE al.name = TRIM(SUBSTRING_INDEX(cr.color, '/', 1))
AND al.id_lang = 1
AND a.id_attribute_group = 1
ORDER BY a.id_attribute ASC
LIMIT 1
) AS color_name,
(
SELECT a.id_attribute
FROM ps_attribute_lang al
JOIN ps_attribute a
ON a.id_attribute = al.id_attribute
JOIN attribute_group ag
ON ag.id_attribute_group = a.id_attribute_group
WHERE al.name = REPLACE(TRIM(cr.combination), ',', '.')
AND al.id_lang = 1
ORDER BY a.id_attribute ASC
LIMIT 1
) AS combi_attribute_id,
(
SELECT al.name
FROM ps_attribute_lang al
JOIN ps_attribute a
ON a.id_attribute = al.id_attribute
JOIN attribute_group ag
ON ag.id_attribute_group = a.id_attribute_group
WHERE al.name = REPLACE(TRIM(cr.combination), ',', '.')
AND al.id_lang = 1
ORDER BY a.id_attribute ASC
LIMIT 1
) AS combi_name
FROM catalog_rows cr
),
existing_combinations AS (
SELECT
ma.id_product_catalog,
pa.id_product_attribute,
pa.ean13
FROM matched_attributes ma
JOIN ps_product_attribute_combination pac_color
ON pac_color.id_attribute = ma.color_attribute_id
JOIN ps_product_attribute_combination pac_combi
ON pac_combi.id_product_attribute = pac_color.id_product_attribute
AND pac_combi.id_attribute = ma.combi_attribute_id
JOIN ps_product_attribute pa
ON pa.id_product_attribute = pac_color.id_product_attribute
AND pa.id_product = ${safeProductId}
)
SELECT
cr.id_product_catalog,
cr.ean_13 AS catalog_ean13,
ma.color_attribute_id,
ma.color_name,
ma.combi_attribute_id,
ma.combi_name,
ec.id_product_attribute,
ec.ean13 AS nine_b_ean13,
ROUND(cr.wholesale_price * (1 - (CAST(cr.voc_discount AS DECIMAL(10, 4)) / 100)), 0)
AS nine_b_voc,
ROUND(cr.moc_price * (1 - (CAST(cr.max_moc_dicount AS DECIMAL(10, 4)) / 100)), 0)
AS nine_b_moc,
CASE
WHEN cr.skladovka = 1
AND (
ma.color_attribute_id IS NULL
OR ma.combi_attribute_id IS NULL
OR ec.id_product_attribute IS NULL
OR ec.ean13 <> cr.ean_13
)
THEN 1
ELSE 0
END AS has_problem
FROM catalog_rows cr
LEFT JOIN matched_attributes ma
ON ma.id_product_catalog = cr.id_product_catalog
LEFT JOIN existing_combinations ec
ON ec.id_product_catalog = cr.id_product_catalog
ORDER BY cr.id_product_catalog ASC
`;
}
export function suggestedProductsSql({ manufacturerId, supplierReference, productName }) {
const safeManufacturerId = parsePositiveInteger(manufacturerId, "manufacturerId");
const safeSupplierReference = escapeSqlLike(supplierReference);
const safeProductName = escapeSqlLike(productName);
if (!safeSupplierReference && !safeProductName) {
throw new Error("supplierReference or productName is required.");
}
const conditions = [];
if (safeSupplierReference) {
conditions.push(`p.supplier_reference LIKE '%${safeSupplierReference}%'`);
}
if (safeProductName) {
conditions.push(`pl.name LIKE '%${safeProductName}%'`);
}
return `
SELECT
p.id_product,
pl.name,
p.supplier_reference,
p.active,
(
SELECT CONCAT('https://www.9b-plus.com/', i.id_image, '-cart_default/', pli.link_rewrite, '.jpg')
FROM ps_image i
JOIN ps_product_lang pli
ON pli.id_product = i.id_product
AND pli.id_lang = 2
WHERE i.id_product = p.id_product
ORDER BY i.cover DESC, i.position ASC
LIMIT 1
) AS image_url
FROM ps_product p
JOIN ps_product_lang pl
ON pl.id_product = p.id_product
AND pl.id_lang = 1
WHERE p.id_manufacturer = ${safeManufacturerId}
AND (${conditions.join(" OR ")})
ORDER BY p.active DESC, p.id_product DESC
LIMIT 20
`;
}
export function mappingSourceTablesSql() {
return `
SELECT table_name
FROM information_schema.tables
WHERE table_schema = DATABASE()
AND table_name IN (
'ps_product_catalog_source',
'ps_product_catalog_manufacturer_source'
)
ORDER BY FIELD(
table_name,
'ps_product_catalog_manufacturer_source',
'ps_product_catalog_source'
)
`;
}
export function mappingSourcesSql({ manufacturerId }) {
const safeManufacturerId = parsePositiveInteger(manufacturerId, "manufacturerId");
return `
SELECT
ms.id_product_catalog_manufacturer_source AS mapping_id,
ms.id_manufacturer,
ms.id_product_catalog_source,
ms.enabled AS mapping_enabled,
ms.priority AS mapping_priority,
s.source_key,
s.source_name,
s.source_type,
s.base_url,
s.enabled,
s.priority AS source_priority,
s.search_primary_key,
s.search_fallback_keys,
s.fallback_source_type,
s.search_url_template,
s.fallback_url_template,
s.search_context_keys,
s.access_mode,
s.browser_engine,
s.browser_headless,
s.wait_after_load_ms,
s.note
FROM ps_product_catalog_manufacturer_source ms
JOIN ps_product_catalog_source s
ON s.id_product_catalog_source = ms.id_product_catalog_source
WHERE ms.id_manufacturer = ${safeManufacturerId}
ORDER BY
ms.priority ASC,
s.priority ASC,
s.source_name ASC
`;
}
function escapeSqlString(value) {
return String(value ?? "").replaceAll("\\", "\\\\").replaceAll("'", "''").trim();
}
function escapeSqlLike(value) {
return escapeSqlString(value).replaceAll("%", "\\%").replaceAll("_", "\\_");
}
function parsePositiveInteger(value, name) {
const parsed = Number.parseInt(value, 10);
if (!Number.isInteger(parsed) || parsed <= 0) {
throw new Error(`${name} must be a positive integer.`);
}
return parsed;
}
function normalizeEan(value) {
return String(value ?? "").replace(/\D/g, "");
}
+62
View File
@@ -0,0 +1,62 @@
import fs from "node:fs";
import { info, warn } from "../lib/log.js";
const steps = [
{
id: "load-context",
title: "Load workbook and local configuration",
status: "planned",
},
{
id: "load-manufacturer",
title: "Load selected manufacturer and products to process",
status: "needs-mapping",
},
{
id: "check-stock",
title: "Check supplier stock and existing catalog combinations",
status: "needs-mapping",
},
{
id: "build-combinations",
title: "Create or update product combinations",
status: "needs-mapping",
},
{
id: "update-prices",
title: "Update product and combination prices",
status: "needs-mapping",
},
{
id: "update-texts",
title: "Generate/update product card texts",
status: "needs-mapping",
},
{
id: "handle-pictures",
title: "Resolve cover/detail product pictures",
status: "needs-mapping",
},
];
export async function runCatalogMaker({ config }) {
info("Catalog maker workflow");
info(`Mode: ${config.dryRun ? "dry-run" : "live"}`);
info(`Workbook: ${config.sourceWorkbook}`);
if (!fs.existsSync(config.sourceWorkbook)) {
throw new Error(`Source workbook not found: ${config.sourceWorkbook}`);
}
info("");
info("Planned replacement for Excel button: Catalog maker - 20");
for (const step of steps) {
const marker = step.status === "planned" ? "ready" : step.status;
info(`- ${step.id}: ${marker} - ${step.title}`);
}
warn("");
warn("No database or catalog changes were made. Database access in this project is read-only by rule #1.");
warn("Next step: map the original VBA/SQL behavior for CommandButton20_Click.");
}
+15
View File
@@ -0,0 +1,15 @@
import test from "node:test";
import assert from "node:assert/strict";
import { loadConfig } from "../src/config.js";
test("defaults to dry-run mode", () => {
const config = loadConfig({ configPath: "missing.json" });
assert.equal(config.dryRun, true);
});
test("cli dry-run override wins over local config", () => {
const config = loadConfig({ configPath: "missing.json", dryRun: true });
assert.equal(config.dryRun, true);
});
+79
View File
@@ -0,0 +1,79 @@
import test from "node:test";
import assert from "node:assert/strict";
import { assertReadOnlySql } from "../src/lib/sql-readonly.js";
import { assertAllowedWriteSql } from "../src/lib/sql-write-allowlist.js";
import {
catalogMakerSql,
mappingSourceTablesSql,
mappingSourcesSql,
prepareManufacturerCatalogSqls,
productByEanSql,
productCombinationInfoSql,
productCombinationsByReferenceSql,
productToDoByManufacturerSql,
} from "../src/sql/catalog-maker.js";
test("manufacturer query is read-only", () => {
assert.doesNotThrow(() => assertReadOnlySql(catalogMakerSql.manufacturers));
});
test("languages query is read-only", () => {
assert.doesNotThrow(() => assertReadOnlySql(catalogMakerSql.languages));
});
test("product to-do query is read-only", () => {
assert.doesNotThrow(() =>
assertReadOnlySql(productToDoByManufacturerSql({ manufacturerId: 13, position: 1 })),
);
});
test("product combinations query is read-only", () => {
assert.doesNotThrow(() =>
assertReadOnlySql(
productCombinationsByReferenceSql({ manufacturerId: 13, supplierReference: "TEST" }),
),
);
});
test("product combination info query is read-only", () => {
assert.doesNotThrow(() =>
assertReadOnlySql(
productCombinationInfoSql({
productId: 135520,
manufacturerId: 13,
supplierReference: "ZFHS099",
}),
),
);
});
test("product by EAN query is read-only", () => {
assert.doesNotThrow(() =>
assertReadOnlySql(productByEanSql({ manufacturerId: 13, ean: "8058428191321" })),
);
});
test("mapping source queries are read-only", () => {
assert.doesNotThrow(() => assertReadOnlySql(mappingSourceTablesSql()));
assert.doesNotThrow(() =>
assertReadOnlySql(mappingSourcesSql({ manufacturerId: 13 })),
);
});
test("manufacturer refresh updates are narrowly allowlisted", () => {
for (const sql of prepareManufacturerCatalogSqls({ manufacturerId: 13 })) {
assert.doesNotThrow(() =>
assertAllowedWriteSql(sql, { operation: "catalog-maker-refresh-manufacturer" }),
);
}
});
test("random update remains blocked", () => {
assert.throws(
() =>
assertAllowedWriteSql("UPDATE ps_product SET price = 0 WHERE id_product = 1", {
operation: "catalog-maker-refresh-manufacturer",
}),
/allowlist/,
);
});
+21
View File
@@ -0,0 +1,21 @@
import test from "node:test";
import assert from "node:assert/strict";
import { assertReadOnlySql } from "../src/lib/sql-readonly.js";
test("allows select statements", () => {
assert.doesNotThrow(() => assertReadOnlySql("SELECT * FROM ps_product LIMIT 1"));
});
test("blocks update statements", () => {
assert.throws(
() => assertReadOnlySql("UPDATE ps_product SET price = 0"),
/Blocked non-read-only SQL statement: UPDATE/,
);
});
test("blocks insert statements after a leading comment", () => {
assert.throws(
() => assertReadOnlySql("-- old macro query\nINSERT INTO ps_product VALUES (1)"),
/Blocked non-read-only SQL statement: INSERT/,
);
});