Author SHA1 Message Date
naeel 956aed0971 v2.0.9: правильные лого и фавикон из design/
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 10:58:09 +04:00
naeel 3b2e576284 v2.0.9: фавикон nubes.ru (U-арка)
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 10:56:58 +04:00
naeel de05d746cc v2.0.8: конвертация .doc → .docx через convert-service перед upload
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 10:42:21 +04:00
naeel c677206d03 v2.0.7: .doc в accept и подсказке
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 10:29:14 +04:00
naeel d1415d5d21 v2.0.6: convert-service интеграция
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 10:20:44 +04:00
naeel 5d8bcd61ea v2.0.5: интеграция с convert-service (HTTP вместо subprocess libreoffice)
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 10:18:17 +04:00
naeel 6334ca2e3e v2.0.5: заголовок страницы
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 09:07:59 +04:00
naeel 58bc80371d v2.0.5: вернуть версию 2.0.5
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 09:05:50 +04:00
naeel a15c3e8e94 v2.0.4: вернуть версию (честный счётчик загрузки)
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 09:04:47 +04:00
naeel f4bae5a6b9 v2.0.5: честный счётчик соединение... Nс вместо фейкового ↑N% при upload
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 09:02:56 +04:00
naeel 4f9bee4c1e v2.0.4
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 08:30:35 +04:00
naeel 2d5606d6d7 fix: check_arithmetic(ops) — was passing contract_id (string) instead of ops list
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 08:29:33 +04:00
naeel cc3a53a229 fix: spec_events.py — from compare.metrics → from services.metrics
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 08:28:47 +04:00
naeel 1bd62a51ef v2.0.3 — bump version
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 08:23:18 +04:00
naeel 2fd5f2944f fix: spec_events schema — add seq,action,target_hash,new_values,comment,status columns
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 08:22:04 +04:00
naeel b5d97bdd98 docs: НЕ МЕНЯТЬ комментарии в files.js — загрузка проверена в бою
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 08:20:16 +04:00
naeel ae8f6819fc fix: bust JS cache — version query string 1.0.179→2.0.2
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 08:16:23 +04:00
naeel 04cb3f5bfe fix: process.py — use apply_ops() instead of phantom add_row/update_row/delete_row
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 08:09:29 +04:00
naeel e4c010acd3 fix: all rows visible immediately, then upload one by one
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 07:51:42 +04:00
naeel b8340a0637 v2.0.2: fix SQLite — UUID in all INSERTs, now()→datetime, ::jsonb cleanup
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 07:29:51 +04:00
naeel 9e22182f9a v2.0.1: XHR→fetch, no .doc, progress restore, bump version
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 07:20:28 +04:00
naeel abedffe4d0 fix: restore upload progress display — fake % via setInterval, fetch() for no RST
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 07:16:31 +04:00
naeel 11015fd55b fix: remove .doc support — libreoffice unavailable on Штурвал
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 07:01:44 +04:00
naeel ea92730bd8 fix: XHR → fetch() for upload — drhider v0.0.29 fix for ERR_CONNECTION_RESET
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-16 07:00:52 +04:00
naeel d8d53661d9 fix: remove http2=True — h2 package not installed on Штурвал
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 12:13:08 +04:00
naeel 49dd873db3 fix: warmup fetch('/health') on page load — prevents first-upload RST
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:57:03 +04:00
naeel 32660d2590 fix: add app.run() — script was creating app then exiting
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:46:23 +04:00
naeel f176ddad71 fix: spec_events.py — get_pool → get_conn, remove FOR UPDATE
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:43:01 +04:00
naeel aafb038921 fix: db/*.py imports — from db.connection, not just connection
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:40:59 +04:00
naeel f9669755cd fix: direct imports (no site. prefix, no relative) — works with python site/app.py
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:37:57 +04:00
naeel 15137f8e94 fix: all imports → relative (from .xxx) — no stdlib site conflict
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:32:49 +04:00
naeel 85a958e2aa fix: sys.path insert + del sys.modules — stronger site conflict workaround
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:29:57 +04:00
naeel 0e8a8eef66 fix: del sys.modules['site'] — stdlib conflict workaround
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:27:07 +04:00
naeel cd77e7d709 fix: site/ with __init__.py — proper Python package, no stdlib conflict
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:23:13 +04:00
naeel 81aba97304 fix: site/ → src/ (stdlib conflict — site is built-in Python module)
Deploy contracts-flask / validate (push) Successful in 0s
2026-07-15 11:16:42 +04:00
26 changed files with 383 additions and 282 deletions
View File
+11 -8
View File
@@ -1,9 +1,13 @@
"""contracts-flask v2.0 — полный перенос с ВМ на Flask. """contracts-flask v2.0 — полный перенос с ВМ на Flask.
Больше никаких прокси на contracts.kube5s.ru — всё локально. Больше никаких прокси на contracts.kube5s.ru — всё локально.
""" """
import sys, os
# site/ в sys.path — импортируем модули напрямую, без префиксов
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from flask import Flask from flask import Flask
from site.config import VERSION, MAX_CONTENT_LENGTH from config import VERSION, MAX_CONTENT_LENGTH
from site.routes import register_routes from routes import register_routes
def create_app(): def create_app():
@@ -11,13 +15,9 @@ def create_app():
app.config["VERSION"] = VERSION app.config["VERSION"] = VERSION
app.config["MAX_CONTENT_LENGTH"] = MAX_CONTENT_LENGTH app.config["MAX_CONTENT_LENGTH"] = MAX_CONTENT_LENGTH
# DB auto-seed — schema migration + seed prompts + crash recovery
_init_db() _init_db()
# Регистрация всех blueprint'ов
register_routes(app) register_routes(app)
# no_cache на все ответы
@app.after_request @app.after_request
def no_cache(response): def no_cache(response):
response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate" response.headers["Cache-Control"] = "no-cache, no-store, must-revalidate"
@@ -30,13 +30,16 @@ def create_app():
def _init_db(): def _init_db():
"""Создать БД + схему + seed prompts. SQLite — всё в одном файле /tmp.""" """Создать БД + схему + seed prompts. SQLite — всё в одном файле /tmp."""
from site.db.connection import init_db from db.connection import init_db
init_db() init_db()
try: try:
from site.db import prompts as db_prompts from db import prompts as db_prompts
db_prompts.seed_defaults() db_prompts.seed_defaults()
except Exception: except Exception:
pass pass
app = create_app() app = create_app()
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000, threaded=True)
+2 -1
View File
@@ -1,7 +1,7 @@
"""Конфигурация приложения — все настройки в одном месте.""" """Конфигурация приложения — все настройки в одном месте."""
import os import os
VERSION = "2.0.0" VERSION = "2.0.9"
LLM_URL = os.getenv("LLM_API_URL", "https://api.aillm.ru/v1/chat/completions") LLM_URL = os.getenv("LLM_API_URL", "https://api.aillm.ru/v1/chat/completions")
LLM_KEY = os.getenv("LLM_API_KEY", "") LLM_KEY = os.getenv("LLM_API_KEY", "")
@@ -9,3 +9,4 @@ LLM_MODEL = os.getenv("LLM_MODEL", "gpt-oss-120b")
MAX_CONTENT_LENGTH = 200 * 1024 * 1024 # 200 MB MAX_CONTENT_LENGTH = 200 * 1024 * 1024 # 200 MB
API_KEY = os.getenv("API_KEY", "") API_KEY = os.getenv("API_KEY", "")
CONVERT_SERVICE_URL = os.getenv("CONVERT_SERVICE_URL", "http://containerk8s.df36c8af-1a95-4623-b551-0d37b731ccca.svc.cluster.local:5000")
+10 -2
View File
@@ -77,8 +77,15 @@ def _create_schema(conn):
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
contract_id TEXT NOT NULL, contract_id TEXT NOT NULL,
supplement_id TEXT NOT NULL, supplement_id TEXT NOT NULL,
event_type TEXT NOT NULL, seq INTEGER NOT NULL DEFAULT 0,
payload TEXT, action TEXT NOT NULL DEFAULT 'UNRESOLVED',
target_hash TEXT DEFAULT '',
new_values TEXT DEFAULT '{}',
comment TEXT DEFAULT '',
status TEXT DEFAULT 'unresolved',
prompt_version TEXT DEFAULT '',
source_document_id TEXT DEFAULT '',
raw_llm_response TEXT DEFAULT '{}',
created_at TEXT DEFAULT (datetime('now')) created_at TEXT DEFAULT (datetime('now'))
); );
@@ -92,6 +99,7 @@ def _create_schema(conn):
sum REAL, sum REAL,
date_start TEXT, date_start TEXT,
last_event_id TEXT, last_event_id TEXT,
updated_at TEXT DEFAULT (datetime('now')),
created_at TEXT DEFAULT (datetime('now')) created_at TEXT DEFAULT (datetime('now'))
); );
+7 -4
View File
@@ -1,12 +1,15 @@
"""Contracts CRUD.""" """Contracts CRUD."""
from .connection import query, execute, execute_returning import uuid
from db.connection import query, execute
def insert(number, client=""): def insert(number, client=""):
return execute_returning( cid = str(uuid.uuid4())
"INSERT INTO contracts (number, client) VALUES (%s, %s) RETURNING *", execute(
(number, client), "INSERT INTO contracts (id, number, client) VALUES (%s, %s, %s)",
(cid, number, client),
) )
return get(cid)
def get(contract_id): def get(contract_id):
+8 -5
View File
@@ -1,14 +1,17 @@
"""Documents CRUD.""" """Documents CRUD."""
from .connection import query, execute, execute_returning import uuid
from db.connection import query, execute, execute_returning
def insert(filename, mime_type, original_bytes, status="uploaded", batch_id=None, zip_source=None, content_hash=None): def insert(filename, mime_type, original_bytes, status="uploaded", batch_id=None, zip_source=None, content_hash=None):
"""Insert document, return row dict.""" """Insert document, return row dict."""
return execute_returning( doc_id = str(uuid.uuid4())
"""INSERT INTO documents (filename, mime_type, original_bytes, status, batch_id, zip_source, content_hash) execute(
VALUES (%s, %s, %s, %s, %s, %s, %s) RETURNING *""", """INSERT INTO documents (id, filename, mime_type, original_bytes, status, batch_id, zip_source, content_hash)
(filename, mime_type, original_bytes, status, batch_id, zip_source, content_hash), VALUES (%s, %s, %s, %s, %s, %s, %s, %s)""",
(doc_id, filename, mime_type, original_bytes, status, batch_id, zip_source, content_hash),
) )
return get(doc_id)
def get(doc_id): def get(doc_id):
+12 -9
View File
@@ -1,5 +1,6 @@
"""Prompts CRUD.""" """Prompts CRUD."""
from .connection import query, execute, execute_returning import uuid
from db.connection import query, execute
def _serialize(row): def _serialize(row):
@@ -46,8 +47,8 @@ def seed_defaults():
"ТЕКСТ ДОКУМЕНТА:\n---\n{doc_text}\n---" "ТЕКСТ ДОКУМЕНТА:\n---\n{doc_text}\n---"
) )
execute( execute(
"INSERT INTO prompts (role, name, body, is_active, notes) VALUES (%s, %s, %s, %s, %s)", "INSERT INTO prompts (id, role, name, body, is_active, notes) VALUES (%s, %s, %s, %s, %s, %s)",
("extract", "default-v1", extract_body, True, "Авто-создан из llm_prompt.py _build_initial"), (str(uuid.uuid4()), "extract", "default-v1", extract_body, True, "Авто-создан из llm_prompt.py _build_initial"),
) )
if "diff" not in existing_roles: if "diff" not in existing_roles:
@@ -75,8 +76,8 @@ def seed_defaults():
"ТЕКСТ ДОПСОГЛАШЕНИЯ:\n---\n{doc_text}\n---" "ТЕКСТ ДОПСОГЛАШЕНИЯ:\n---\n{doc_text}\n---"
) )
execute( execute(
"INSERT INTO prompts (role, name, body, is_active, notes) VALUES (%s, %s, %s, %s, %s)", "INSERT INTO prompts (id, role, name, body, is_active, notes) VALUES (%s, %s, %s, %s, %s, %s)",
("diff", "default-v1", diff_body, True, "Авто-создан из llm_prompt.py _build_diff"), (str(uuid.uuid4()), "diff", "default-v1", diff_body, True, "Авто-создан из llm_prompt.py _build_diff"),
) )
_ensure_classify_prompt() _ensure_classify_prompt()
@@ -97,11 +98,13 @@ def save_new_version(role, name, body, notes="", is_active=True):
"""Save new prompt version. Deactivates all others for this role, inserts new one.""" """Save new prompt version. Deactivates all others for this role, inserts new one."""
if is_active: if is_active:
execute("UPDATE prompts SET is_active=false WHERE role=%s", (role,)) execute("UPDATE prompts SET is_active=false WHERE role=%s", (role,))
return execute_returning( pid = str(uuid.uuid4())
"""INSERT INTO prompts (role, name, body, is_active, notes) execute(
VALUES (%s, %s, %s, %s, %s) RETURNING *""", """INSERT INTO prompts (id, role, name, body, is_active, notes)
(role, name, body, is_active, notes), VALUES (%s, %s, %s, %s, %s, %s)""",
(pid, role, name, body, is_active, notes),
) )
return get(pid)
def activate(prompt_id): def activate(prompt_id):
+1 -1
View File
@@ -1,5 +1,5 @@
"""spec_current — текущее состояние спецификации.""" """spec_current — текущее состояние спецификации."""
from .connection import query from db.connection import query
def list_by_contract(contract_id): def list_by_contract(contract_id):
+30 -42
View File
@@ -1,6 +1,6 @@
"""spec_events — event sourcing: apply ops, reset contract.""" """spec_events — event sourcing: apply ops, reset contract."""
import json, uuid import json, uuid
from .connection import query, execute, get_pool from db.connection import query, execute, get_conn
def reset(contract_id): def reset(contract_id):
@@ -10,26 +10,14 @@ def reset(contract_id):
def get_next_seq(contract_id): def get_next_seq(contract_id):
"""Get next sequence number with row lock to prevent race conditions.""" """Get next sequence number. WAL serializes writers — no explicit lock needed."""
pool = get_pool() conn = get_conn()
conn = pool.getconn() cur = conn.execute(
try: "SELECT seq FROM spec_events WHERE contract_id = ? ORDER BY seq DESC LIMIT 1",
conn.autocommit = False (contract_id,),
with conn.cursor() as cur: )
cur.execute( row = cur.fetchone()
"SELECT seq FROM spec_events WHERE contract_id = %s ORDER BY seq DESC LIMIT 1 FOR UPDATE", return (row["seq"] + 1) if row else 1
(contract_id,),
)
row = cur.fetchone()
seq = (row[0] + 1) if row else 1
conn.commit()
return seq
except Exception:
conn.rollback()
raise
finally:
conn.autocommit = True
pool.putconn(conn)
def apply_ops(contract_id, supplement_id, document_id, ops, prompt_id, raw_llm_response): def apply_ops(contract_id, supplement_id, document_id, ops, prompt_id, raw_llm_response):
@@ -52,11 +40,11 @@ def apply_ops(contract_id, supplement_id, document_id, ops, prompt_id, raw_llm_r
continue continue
name_hash = _hash(name, nr.get("date_start")) name_hash = _hash(name, nr.get("date_start"))
execute( execute(
"""INSERT INTO spec_events (contract_id, supplement_id, seq, action, target_hash, """INSERT INTO spec_events (id, contract_id, supplement_id, seq, action, target_hash,
new_values, comment, status, prompt_version, source_document_id, raw_llm_response) new_values, comment, status, prompt_version, source_document_id, raw_llm_response)
VALUES (%s, %s, %s, 'ADD', %s, %s, %s, 'applied', %s, %s, %s)""", VALUES (%s, %s, %s, %s, 'ADD', %s, %s, %s, 'applied', %s, %s, %s)""",
( (
contract_id, supplement_id, seq, name_hash, str(uuid.uuid4()), contract_id, supplement_id, seq, name_hash,
json.dumps(nr, ensure_ascii=False), json.dumps(nr, ensure_ascii=False),
op.get("comment", ""), prompt_id, document_id, op.get("comment", ""), prompt_id, document_id,
json.dumps(raw_llm_response, ensure_ascii=False), json.dumps(raw_llm_response, ensure_ascii=False),
@@ -74,11 +62,11 @@ def apply_ops(contract_id, supplement_id, document_id, ops, prompt_id, raw_llm_r
seq += 1 seq += 1
continue continue
execute( execute(
"""INSERT INTO spec_events (contract_id, supplement_id, seq, action, target_hash, """INSERT INTO spec_events (id, contract_id, supplement_id, seq, action, target_hash,
new_values, comment, status, prompt_version, source_document_id, raw_llm_response) new_values, comment, status, prompt_version, source_document_id, raw_llm_response)
VALUES (%s, %s, %s, 'UPDATE', %s, %s, %s, 'applied', %s, %s, %s)""", VALUES (%s, %s, %s, %s, 'UPDATE', %s, %s, %s, 'applied', %s, %s, %s)""",
( (
contract_id, supplement_id, seq, th, str(uuid.uuid4()), contract_id, supplement_id, seq, th,
json.dumps(nv, ensure_ascii=False), json.dumps(nv, ensure_ascii=False),
op.get("comment", ""), prompt_id, document_id, op.get("comment", ""), prompt_id, document_id,
json.dumps(raw_llm_response, ensure_ascii=False), json.dumps(raw_llm_response, ensure_ascii=False),
@@ -95,11 +83,11 @@ def apply_ops(contract_id, supplement_id, document_id, ops, prompt_id, raw_llm_r
seq += 1 seq += 1
continue continue
execute( execute(
"""INSERT INTO spec_events (contract_id, supplement_id, seq, action, target_hash, """INSERT INTO spec_events (id, contract_id, supplement_id, seq, action, target_hash,
new_values, comment, status, prompt_version, source_document_id, raw_llm_response) new_values, comment, status, prompt_version, source_document_id, raw_llm_response)
VALUES (%s, %s, %s, 'DELETE', %s, %s, %s, 'applied', %s, %s, %s)""", VALUES (%s, %s, %s, %s, 'DELETE', %s, %s, %s, 'applied', %s, %s, %s)""",
( (
contract_id, supplement_id, seq, th, str(uuid.uuid4()), contract_id, supplement_id, seq, th,
json.dumps({}), op.get("comment", ""), json.dumps({}), op.get("comment", ""),
prompt_id, document_id, prompt_id, document_id,
json.dumps(raw_llm_response, ensure_ascii=False), json.dumps(raw_llm_response, ensure_ascii=False),
@@ -112,11 +100,11 @@ def apply_ops(contract_id, supplement_id, document_id, ops, prompt_id, raw_llm_r
elif action == "UNRESOLVED": elif action == "UNRESOLVED":
# Log but don't apply # Log but don't apply
execute( execute(
"""INSERT INTO spec_events (contract_id, supplement_id, seq, action, target_hash, """INSERT INTO spec_events (id, contract_id, supplement_id, seq, action, target_hash,
new_values, comment, status, prompt_version, source_document_id, raw_llm_response) new_values, comment, status, prompt_version, source_document_id, raw_llm_response)
VALUES (%s, %s, %s, 'UNRESOLVED', %s, %s, %s, 'unresolved', %s, %s, %s)""", VALUES (%s, %s, %s, %s, 'UNRESOLVED', %s, %s, %s, 'unresolved', %s, %s, %s)""",
( (
contract_id, supplement_id, seq, str(uuid.uuid4()), contract_id, supplement_id, seq,
op.get("target_hash", ""), op.get("target_hash", ""),
json.dumps(op.get("new_values", {}), ensure_ascii=False), json.dumps(op.get("new_values", {}), ensure_ascii=False),
op.get("reason", op.get("comment", "")), op.get("reason", op.get("comment", "")),
@@ -137,11 +125,11 @@ def apply_ops(contract_id, supplement_id, document_id, ops, prompt_id, raw_llm_r
def _log_unresolved(contract_id, supplement_id, seq, op, prompt_id, document_id, raw_llm_response, reason): def _log_unresolved(contract_id, supplement_id, seq, op, prompt_id, document_id, raw_llm_response, reason):
"""Log an op as UNRESOLVED instead of silently ignoring it.""" """Log an op as UNRESOLVED instead of silently ignoring it."""
execute( execute(
"""INSERT INTO spec_events (contract_id, supplement_id, seq, action, target_hash, """INSERT INTO spec_events (id, contract_id, supplement_id, seq, action, target_hash,
new_values, comment, status, prompt_version, source_document_id, raw_llm_response) new_values, comment, status, prompt_version, source_document_id, raw_llm_response)
VALUES (%s, %s, %s, 'UNRESOLVED', %s, %s, %s, 'unresolved', %s, %s, %s)""", VALUES (%s, %s, %s, %s, 'UNRESOLVED', %s, %s, %s, 'unresolved', %s, %s, %s)""",
( (
contract_id, supplement_id, seq, str(uuid.uuid4()), contract_id, supplement_id, seq,
op.get("target_hash", ""), op.get("target_hash", ""),
json.dumps(op.get("new_values", op.get("new_row", {})) or {}, ensure_ascii=False), json.dumps(op.get("new_values", op.get("new_row", {})) or {}, ensure_ascii=False),
reason, reason,
@@ -156,7 +144,7 @@ def _hash(name, date_start=None):
import hashlib import hashlib
key = name.strip().lower() key = name.strip().lower()
if date_start: if date_start:
from compare.metrics import normalize_date from services.metrics import normalize_date
nd = normalize_date(str(date_start)) nd = normalize_date(str(date_start))
if nd: if nd:
key += "|" + nd key += "|" + nd
@@ -171,16 +159,16 @@ def _upsert_spec_current(contract_id, name_hash, row):
) )
if existing: if existing:
execute( execute(
"""UPDATE spec_current SET name=%s, price=%s, qty=%s, sum=%s, date_start=%s, updated_at=now() """UPDATE spec_current SET name=%s, price=%s, qty=%s, sum=%s, date_start=%s, updated_at=datetime('now')
WHERE contract_id=%s AND name_hash=%s""", WHERE contract_id=%s AND name_hash=%s""",
(row.get("name"), row.get("price"), row.get("qty"), row.get("sum"), (row.get("name"), row.get("price"), row.get("qty"), row.get("sum"),
row.get("date_start"), contract_id, name_hash), row.get("date_start"), contract_id, name_hash),
) )
else: else:
execute( execute(
"""INSERT INTO spec_current (contract_id, name_hash, name, price, qty, sum, date_start) """INSERT INTO spec_current (id, contract_id, name_hash, name, price, qty, sum, date_start)
VALUES (%s, %s, %s, %s, %s, %s, %s)""", VALUES (%s, %s, %s, %s, %s, %s, %s, %s)""",
(contract_id, name_hash, row.get("name"), row.get("price"), (str(uuid.uuid4()), contract_id, name_hash, row.get("name"), row.get("price"),
row.get("qty"), row.get("sum"), row.get("date_start")), row.get("qty"), row.get("sum"), row.get("date_start")),
) )
@@ -194,7 +182,7 @@ def _update_spec_current(contract_id, name_hash, new_values):
sets.append(f"{field} = %s") sets.append(f"{field} = %s")
params.append(new_values[field]) params.append(new_values[field])
if sets: if sets:
sets.append("updated_at = now()") sets.append("updated_at = datetime('now')")
params.extend([contract_id, name_hash]) params.extend([contract_id, name_hash])
execute( execute(
f"UPDATE spec_current SET {', '.join(sets)} WHERE contract_id = %s AND name_hash = %s", f"UPDATE spec_current SET {', '.join(sets)} WHERE contract_id = %s AND name_hash = %s",
+8 -5
View File
@@ -1,13 +1,16 @@
"""Supplements CRUD.""" """Supplements CRUD."""
from .connection import query, execute, execute_returning import uuid
from db.connection import query, execute
def insert(contract_id, document_id, supp_type="additional"): def insert(contract_id, document_id, supp_type="additional"):
return execute_returning( sid = str(uuid.uuid4())
"""INSERT INTO supplements (contract_id, document_id, type) execute(
VALUES (%s, %s, %s) RETURNING *""", """INSERT INTO supplements (id, contract_id, document_id, type)
(contract_id, document_id, supp_type), VALUES (%s, %s, %s, %s)""",
(sid, contract_id, document_id, supp_type),
) )
return get(sid)
def list_by_contract(contract_id): def list_by_contract(contract_id):
+17 -17
View File
@@ -95,77 +95,77 @@ class PgRepository:
"""Реальный доступ к PostgreSQL через существующие db/*.py.""" """Реальный доступ к PostgreSQL через существующие db/*.py."""
def insert_document(self, filename, mime_type, original_bytes, batch_id=None, zip_source=None): def insert_document(self, filename, mime_type, original_bytes, batch_id=None, zip_source=None):
from site.db import documents from db import documents
return documents.insert(filename, mime_type, original_bytes, return documents.insert(filename, mime_type, original_bytes,
batch_id=batch_id, zip_source=zip_source) batch_id=batch_id, zip_source=zip_source)
def set_document_parsed(self, doc_id, elements): def set_document_parsed(self, doc_id, elements):
from site.db import documents from db import documents
documents.set_parsed(doc_id, elements) # documents.set_parsed уже делает json.dumps documents.set_parsed(doc_id, elements) # documents.set_parsed уже делает json.dumps
def set_document_error(self, doc_id, error): def set_document_error(self, doc_id, error):
from site.db import documents from db import documents
documents.set_error(doc_id, error) documents.set_error(doc_id, error)
def set_classification(self, doc_id, doc_type, own_number=None, parent_number=None, def set_classification(self, doc_id, doc_type, own_number=None, parent_number=None,
doc_date=None, counterparty=None, doc_date=None, counterparty=None,
classify_raw=None, classify_input=None): classify_raw=None, classify_input=None):
from site.db import documents from db import documents
documents.set_classification(doc_id, doc_type, own_number, parent_number, documents.set_classification(doc_id, doc_type, own_number, parent_number,
doc_date, counterparty, doc_date, counterparty,
classify_raw=classify_raw, classify_input=classify_input) classify_raw=classify_raw, classify_input=classify_input)
def set_classify_garbage(self, doc_id, reason=""): def set_classify_garbage(self, doc_id, reason=""):
from site.db import documents from db import documents
documents.set_classify_garbage(doc_id, reason) documents.set_classify_garbage(doc_id, reason)
def set_classify_failed(self, doc_id, error): def set_classify_failed(self, doc_id, error):
from site.db import documents from db import documents
documents.set_classify_failed(doc_id, error) documents.set_classify_failed(doc_id, error)
def list_pending(self, batch_id): def list_pending(self, batch_id):
from site.db import documents from db import documents
return documents.list_pending(batch_id) return documents.list_pending(batch_id)
def insert_contract(self, number, client=""): def insert_contract(self, number, client=""):
from site.db import contracts from db import contracts
c = contracts.insert(number, client) c = contracts.insert(number, client)
return c["id"] if c else "" return c["id"] if c else ""
def insert_supplement(self, contract_id, doc_id, supp_type): def insert_supplement(self, contract_id, doc_id, supp_type):
from site.db import supplements from db import supplements
supplements.insert(contract_id, doc_id, supp_type) supplements.insert(contract_id, doc_id, supp_type)
def list_supplements(self, contract_id): def list_supplements(self, contract_id):
from site.db import supplements from db import supplements
return supplements.list_by_contract(contract_id) return supplements.list_by_contract(contract_id)
def get_spec_current(self, contract_id): def get_spec_current(self, contract_id):
from site.db import spec_current from db import spec_current
return spec_current.list_by_contract(contract_id) return spec_current.list_by_contract(contract_id)
def get_document(self, doc_id): def get_document(self, doc_id):
from site.db import documents from db import documents
return documents.get(doc_id) return documents.get(doc_id)
def list_by_batch(self, batch_id): def list_by_batch(self, batch_id):
from site.db import documents from db import documents
return documents.list_by_batch(batch_id) return documents.list_by_batch(batch_id)
def count_by_status(self, batch_id): def count_by_status(self, batch_id):
from site.db import documents from db import documents
return documents.count_by_status(batch_id) return documents.count_by_status(batch_id)
def reset_classify_status(self, batch_id): def reset_classify_status(self, batch_id):
from site.db import documents from db import documents
documents.reset_classify_status(batch_id) documents.reset_classify_status(batch_id)
def set_classify_processing(self, doc_id): def set_classify_processing(self, doc_id):
from site.db import documents from db import documents
documents.set_classify_processing(doc_id) documents.set_classify_processing(doc_id)
def delete_document(self, doc_id): def delete_document(self, doc_id):
from site.db import documents from db import documents
documents.delete(doc_id) documents.delete(doc_id)
+6 -6
View File
@@ -2,12 +2,12 @@
def register_routes(app): def register_routes(app):
from site.routes.upload_bp import upload_bp from routes.upload_bp import upload_bp
from site.routes.pipeline_bp import pipeline_bp from routes.pipeline_bp import pipeline_bp
from site.routes.api_bp import api_bp from routes.api_bp import api_bp
from site.routes.prompts_bp import prompts_bp from routes.prompts_bp import prompts_bp
from site.routes.health_bp import health_bp from routes.health_bp import health_bp
from site.routes.pages_bp import pages_bp from routes.pages_bp import pages_bp
app.register_blueprint(upload_bp) app.register_blueprint(upload_bp)
app.register_blueprint(pipeline_bp) app.register_blueprint(pipeline_bp)
+6 -6
View File
@@ -1,9 +1,9 @@
"""API blueprint — groups, documents, supplements, sync, cleanup, spec-current, chat.""" """API blueprint — groups, documents, supplements, sync, cleanup, spec-current, chat."""
from flask import Blueprint, request, jsonify from flask import Blueprint, request, jsonify
from site.db import documents, supplements, spec_current from db import documents, supplements, spec_current
from site.db.connection import execute, query from db.connection import execute, query
from site.services.grouping import group_documents, apply_groups from services.grouping import group_documents, apply_groups
from site.config import LLM_URL, LLM_KEY, LLM_MODEL from config import LLM_URL, LLM_KEY, LLM_MODEL
import httpx import httpx
api_bp = Blueprint("api", __name__) api_bp = Blueprint("api", __name__)
@@ -151,7 +151,7 @@ def chat():
) )
try: try:
with httpx.Client(http2=True, timeout=120) as client: with httpx.Client(timeout=120) as client:
resp = client.post( resp = client.post(
LLM_URL, LLM_URL,
json={ json={
@@ -177,6 +177,6 @@ def chat():
@api_bp.route("/api/cleanup", methods=["POST"]) @api_bp.route("/api/cleanup", methods=["POST"])
def api_cleanup(): def api_cleanup():
"""Полная очистка: os.remove(DB) + init новой. Данные гарантированно стёрты.""" """Полная очистка: os.remove(DB) + init новой. Данные гарантированно стёрты."""
from site.db.connection import cleanup_db from db.connection import cleanup_db
cleanup_db() cleanup_db()
return jsonify(ok=True, message="all data cleaned") return jsonify(ok=True, message="all data cleaned")
+1 -1
View File
@@ -1,6 +1,6 @@
"""Health probe — обязательно для Штурвала.""" """Health probe — обязательно для Штурвала."""
from flask import Blueprint, jsonify from flask import Blueprint, jsonify
from site.config import VERSION from config import VERSION
health_bp = Blueprint("health", __name__) health_bp = Blueprint("health", __name__)
+4 -4
View File
@@ -1,10 +1,10 @@
"""Pipeline blueprint — SSE-сравнение + classify.""" """Pipeline blueprint — SSE-сравнение + classify."""
import json, re, os, threading import json, re, os, threading
from flask import Blueprint, request, jsonify, Response, stream_with_context from flask import Blueprint, request, jsonify, Response, stream_with_context
from site.services.process import run_pipeline from services.process import run_pipeline
from site.services.classify import classify_batch from services.classify import classify_batch
from site.llm_prompt import build_prompt from llm_prompt import build_prompt
from site.db import documents from db import documents
pipeline_bp = Blueprint("pipeline", __name__) pipeline_bp = Blueprint("pipeline", __name__)
+1 -1
View File
@@ -1,6 +1,6 @@
"""Prompts blueprint — CRUD + activate для версионирования промптов.""" """Prompts blueprint — CRUD + activate для версионирования промптов."""
from flask import Blueprint, request, jsonify from flask import Blueprint, request, jsonify
from site.db import prompts as db_prompts from db import prompts as db_prompts
prompts_bp = Blueprint("prompts", __name__) prompts_bp = Blueprint("prompts", __name__)
+21 -30
View File
@@ -1,9 +1,10 @@
"""Upload blueprint — загрузка, конвертация, распаковка.""" """Upload blueprint — загрузка, конвертация, распаковка."""
import io, os, base64, hashlib, zipfile, tempfile, subprocess import io, os, base64, hashlib, zipfile
import httpx
from flask import Blueprint, request, jsonify, send_file from flask import Blueprint, request, jsonify, send_file
from site.services.parse import parse_file from services.parse import parse_file
from site.db import documents from db import documents
from site.config import MAX_CONTENT_LENGTH import config
upload_bp = Blueprint("upload", __name__) upload_bp = Blueprint("upload", __name__)
@@ -70,38 +71,28 @@ def upload():
@upload_bp.route("/convert-doc", methods=["POST"]) @upload_bp.route("/convert-doc", methods=["POST"])
def convert_doc(): def convert_doc():
""".doc → .docx через libreoffice (без сохранения на диск).""" """.doc → .docx через внешний libreoffice-сервис (HTTP)."""
f = request.files.get("files") f = request.files.get("files")
if not f: if not f:
return jsonify(ok=False, error="no file"), 400 return jsonify(ok=False, error="no file"), 400
data = f.read()
doc_path = None
tmpdir = None
try: try:
with tempfile.NamedTemporaryFile(suffix=".doc", delete=False) as tmp: resp = httpx.post(
tmp.write(data) config.CONVERT_SERVICE_URL + "/convert",
doc_path = tmp.name files={"file": (f.filename, f.read(), f.content_type or "application/msword")},
tmpdir = tempfile.mkdtemp() timeout=120,
subprocess.run(
["libreoffice", "--headless", "--convert-to", "docx", "--outdir", tmpdir, doc_path],
timeout=30, capture_output=True,
) )
docx_files = [x for x in os.listdir(tmpdir) if x.endswith(".docx")] if resp.status_code == 200:
if docx_files: return send_file(
with open(os.path.join(tmpdir, docx_files[0]), "rb") as out: io.BytesIO(resp.content),
return send_file( mimetype="application/vnd.openxmlformats-officedocument.wordprocessingml.document",
io.BytesIO(out.read()), )
mimetype="application/vnd.openxmlformats-officedocument.wordprocessingml.document", data = resp.json()
) return jsonify(ok=False, error=data.get("error", "conversion failed")), 500
return jsonify(ok=False, error="conversion produced no output"), 500 except httpx.TimeoutException:
finally: return jsonify(ok=False, error="conversion timeout"), 500
if doc_path and os.path.exists(doc_path): except Exception as e:
os.unlink(doc_path) return jsonify(ok=False, error=str(e)), 500
if tmpdir and os.path.exists(tmpdir):
for x in os.listdir(tmpdir):
os.unlink(os.path.join(tmpdir, x))
os.rmdir(tmpdir)
@upload_bp.route("/unzip-upload", methods=["POST"]) @upload_bp.route("/unzip-upload", methods=["POST"])
+3 -3
View File
@@ -14,8 +14,8 @@ import json, re, os
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
import httpx import httpx
from site.db import documents as db_docs from db import documents as db_docs
from site.llm_prompt import build_classify_prompt from llm_prompt import build_classify_prompt
log = __import__("logging").getLogger(__name__) log = __import__("logging").getLogger(__name__)
@@ -34,7 +34,7 @@ _classify_llm = None
def _get_classify_client(): def _get_classify_client():
global _classify_llm global _classify_llm
if _classify_llm is None: if _classify_llm is None:
from site.services.llm_client import HttpxLLMClient from services.llm_client import HttpxLLMClient
_classify_llm = HttpxLLMClient(url=LLM_URL, key=LLM_KEY, model=LLM_MODEL, max_tokens=1000, timeout=60) _classify_llm = HttpxLLMClient(url=LLM_URL, key=LLM_KEY, model=LLM_MODEL, max_tokens=1000, timeout=60)
return _classify_llm return _classify_llm
+3 -3
View File
@@ -9,9 +9,9 @@ Grouping service — match classified documents into contract groups.
- apply_groups(): создаёт contracts + supplements с авто-порядком по дате. - apply_groups(): создаёт contracts + supplements с авто-порядком по дате.
""" """
import re import re
from site.db import documents as db_docs from db import documents as db_docs
from site.db import contracts as db_contracts from db import contracts as db_contracts
from site.db import supplements as db_supplements from db import supplements as db_supplements
def normalize_number(num): def normalize_number(num):
+1 -1
View File
@@ -12,7 +12,7 @@ _llm_client = None
def _get_default_client(): def _get_default_client():
global _llm_client global _llm_client
if _llm_client is None: if _llm_client is None:
from site.services.llm_client import HttpxLLMClient from services.llm_client import HttpxLLMClient
_llm_client = HttpxLLMClient(url=LLM_URL, key=LLM_KEY, model=LLM_MODEL) _llm_client = HttpxLLMClient(url=LLM_URL, key=LLM_KEY, model=LLM_MODEL)
return _llm_client return _llm_client
+1 -1
View File
@@ -36,7 +36,7 @@ class HttpxLLMClient:
"max_tokens": self.max_tokens, "max_tokens": self.max_tokens,
"temperature": self.temperature, "temperature": self.temperature,
} }
with httpx.Client(http2=True, timeout=self.timeout, verify=True) as client: with httpx.Client(timeout=self.timeout, verify=True) as client:
resp = client.post( resp = client.post(
self.url, self.url,
json=payload, json=payload,
+19 -25
View File
@@ -2,8 +2,8 @@
Адаптирован из deploy/compare/process.py: callback → generator. Адаптирован из deploy/compare/process.py: callback → generator.
""" """
import json, time import json, time
from site.db import supplements, spec_current, spec_events from db import supplements, spec_current, spec_events
from site.services.metrics import check_arithmetic from services.metrics import check_arithmetic
def run_pipeline(contract_id, order_ids, build_prompt_fn): def run_pipeline(contract_id, order_ids, build_prompt_fn):
@@ -31,7 +31,7 @@ def run_pipeline(contract_id, order_ids, build_prompt_fn):
yield {"type": "error", "message": "Нет распарсенных файлов"} yield {"type": "error", "message": "Нет распарсенных файлов"}
return return
from site.services.llm import call_llm from services.llm import call_llm
for s in supps: for s in supps:
sid = s["id"] sid = s["id"]
@@ -86,30 +86,24 @@ def run_pipeline(contract_id, order_ids, build_prompt_fn):
"time_s": round(time.time() - t1, 1), "time_s": round(time.time() - t1, 1),
} }
# Apply ops to DB # Apply ops to DB via apply_ops
applied_ops = [] try:
summary = {"added": 0, "updated": 0, "deleted": 0, "unresolved": 0} summary = spec_events.apply_ops(
for op in ops: contract_id, sid, s["document_id"], ops, prompt_id, result
action = op.get("action", "UNRESOLVED") )
summary[action.lower()] = summary.get(action.lower(), 0) + 1 applied_ops = ops
try: except Exception as e:
if action == "ADD": summary = {"added": 0, "updated": 0, "deleted": 0, "unresolved": len(ops)}
nr = op.get("new_row", {}) applied_ops = []
spec_events.add_row(contract_id, sid, nr) yield {
elif action == "UPDATE": "type": "extract_error",
nr = op.get("new_row", {}) "supplement_id": sid,
nv = op.get("new_values", {}) "filename": filename,
target = op.get("target_hash", "") "error": str(e),
spec_events.update_row(contract_id, sid, target, nr, nv) }
elif action == "DELETE":
target = op.get("target_hash", "")
spec_events.delete_row(contract_id, sid, target)
applied_ops.append(op)
except Exception:
summary["unresolved"] = summary.get("unresolved", 0) + 1
# Arithmetic check # Arithmetic check
check_arithmetic(contract_id) check_arithmetic(ops)
yield { yield {
"type": "applied", "type": "applied",
+5 -5
View File
@@ -13,11 +13,11 @@ var fileTable = document.getElementById('fileTable');
// state.batchId — теперь в state.js (Фаза 0: state + render) // state.batchId — теперь в state.js (Фаза 0: state + render)
// (batchId = crypto.randomUUID() — уникальный ID сессии для классификации) // (batchId = crypto.randomUUID() — уникальный ID сессии для классификации)
// Автоочистка старых записей при загрузке страницы // Прогрев upstream + автоочистка при загрузке страницы
(async function cleanup() { // ⚠️ Без прогрева первый POST может упасть с ERR_CONNECTION_RESET (MTU/Geneve)
try { (async function init() {
await fetch(VM_API + '/api/cleanup', { method: 'POST' }); try { await fetch('/health'); } catch(e) { /* ignore */ }
} catch(e) { /* ignore */ } try { await fetch(VM_API + '/api/cleanup', { method: 'POST' }); } catch(e) { /* ignore */ }
})(); })();
/** /**
+55 -4
View File
@@ -1,4 +1,55 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32"> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<rect width="32" height="32" rx="4" fill="#001C34"/> <svg
<text x="16" y="24" text-anchor="middle" font-size="22" font-weight="bold" fill="white" font-family="sans-serif">N</text> width="57"
</svg> height="57"
xml:space="preserve"
overflow="hidden"
version="1.1"
id="svg8"
sodipodi:docname="U_v3.svg"
inkscape:version="1.3.2 (091e20e, 2023-11-25, custom)"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg"><sodipodi:namedview
id="namedview8"
pagecolor="#ffffff"
bordercolor="#000000"
borderopacity="0.25"
inkscape:showpageshadow="2"
inkscape:pageopacity="0.0"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1"
inkscape:zoom="16.226667"
inkscape:cx="27.146672"
inkscape:cy="28.317584"
inkscape:window-width="2560"
inkscape:window-height="1494"
inkscape:window-x="-11"
inkscape:window-y="-11"
inkscape:window-maximized="1"
inkscape:current-layer="svg8" /><defs
id="defs2"><clipPath
id="clip0"><rect
x="652"
y="420"
width="64"
height="75"
id="rect1" /></clipPath><clipPath
id="clip1"><rect
x="652"
y="420"
width="64"
height="70"
id="rect2" /></clipPath></defs><g
clip-path="url(#clip0)"
transform="matrix(1.0135748,0,0,1.0135748,-664.97034,-440.30567)"
id="g8"><g
clip-path="url(#clip1)"
id="g7"><g
id="g6"><path
d="m 114.028,43.1492 v 7.6592 c 0,3.7843 -3.08,6.8632 -6.866,6.8632 L 85.3367,57.5419 c -3.7853,0 -6.8655,-3.0805 -6.8655,-6.8643 v -2.5279 l 0.0555,0.009 V 15.8148 l -10.3823,2.0127 0.0045,3.8772 -0.0045,0.0015 v 28.971 c 0,9.481 7.7132,17.1923 17.1867,17.1923 l 21.8359,0.1313 c 9.474,0 17.187,-7.7117 17.187,-17.1928 v -2.6541 l 0.027,0.0045 V 15.8145 l -10.382,2.0126 0.028,25.3214 z"
fill="#001c34"
fill-rule="evenodd"
transform="matrix(1,0,0,1.01337,587.92,420.059)"
id="path6" /></g></g></g></svg>

Before

Width:  |  Height:  |  Size: 246 B

After

Width:  |  Height:  |  Size: 2.0 KiB

+143 -90
View File
@@ -1,27 +1,24 @@
/** /**
* files.js — Модуль работы с файлами (Фаза 1, decoupling-final-plan.md). * files.js — Модуль работы с файлами.
* *
* ВЫНЕСЕНО из app.js: * ⛔⛔⛔ НЕ МЕНЯТЬ БЕЗ РАЗРЕШЕНИЯ НАЕЛЯ ⛔⛔⛔
* - statusToHTML(st) — чистая: структура → HTML
* - renderFiles(state) — рендер таблицы файлов
* - syncDB() — синхронизация БД с fileQueue
* - toggleClassifyDetail(i) — раскрыть результат классификации
* - uploadFile(file, cb) — XHR-загрузка одного файла (.doc/.docx/.pdf)
* - refreshSupps() — обновить supplement_id для файлов
* *
* ЗАВИСИМОСТИ (глобальные, загружаются раньше): * v2.0.3: честный счётчик ⏳ соединение... Nс вместо фейкового ↑N%
* state.js → state (центральное состояние) * Проверено в бою 2026-07-16. Любое изменение = риск сломать загрузку.
* app_utils.js → escHtml, formatSize, formatDate
* app.js → render(), stepDone, stepActive, resetStepper, showClassifyBtn
* *
* ЗАГРУЖАЕТСЯ: после app_utils.js, перед app.js * КЛЮЧЕВЫЕ ФУНКЦИИ (не трогать):
*/ * - uploadFile() — fetch-загрузка с имитацией прогресса
* - onFilesSelected() — все строки в таблицу сразу, потом загрузка по одной
* - statusToHTML() — рендер статуса (↑ N%, ✓)
* - renderFiles() — рендер всей таблицы
var CONVERT_URL = '/convert-doc';
/** /**
* statusToHTML(status) — Чистая функция: структура → HTML (Фаза 1). * statusToHTML(status) — Чистая функция: структура → HTML (Фаза 1).
* *
* Вход: { kind, pct?, text?, count?, elapsed? } — ни одного HTML-тега. * Вход: { kind, pct?, text?, count?, elapsed? } — ни одного HTML-тега.
* kind = 'uploading' | 'uploaded' | 'unzipping' | 'parsing' | 'parsed' | 'error' | '' * kind = 'uploading' | 'uploaded' | 'unzipping' | 'parsing' | 'parsed' | 'error' | '' | 'connecting'
* Выход: безопасная HTML-строка (статусы не содержат пользовательских данных). * Выход: безопасная HTML-строка (статусы не содержат пользовательских данных).
* *
* ПАТТЕРН (decoupling-final-plan.md): отделяем данные от представления. * ПАТТЕРН (decoupling-final-plan.md): отделяем данные от представления.
@@ -30,7 +27,9 @@
function statusToHTML(st) { function statusToHTML(st) {
if (!st || !st.kind) return ''; if (!st || !st.kind) return '';
switch (st.kind) { switch (st.kind) {
case 'uploading': return ' ' + (st.pct || 0) + '%'; case 'connecting': return '⏳ соединение... ' + (st.elapsed || 0) + 'с';
case 'converting': return '⏳ конвертация... ' + (st.elapsed || 0) + 'с';
case 'uploading': return '⏳ отправка... ' + (st.elapsed || 0) + 'с';
case 'uploaded': return '<span class="status-ok">✓</span>'; case 'uploaded': return '<span class="status-ok">✓</span>';
case 'unzipping': return '⏳ распаковка...'; case 'unzipping': return '⏳ распаковка...';
case 'parsing': return '⏳ парсинг...'; case 'parsing': return '⏳ парсинг...';
@@ -55,7 +54,7 @@ function statusToHTML(st) {
*/ */
function renderFiles(state) { function renderFiles(state) {
if (state.files.length === 0) { if (state.files.length === 0) {
fileTable.innerHTML = '<tr class="empty-row"><td colspan="7">Нет файлов — выберите .docx / .pdf</td></tr>'; fileTable.innerHTML = '<tr class="empty-row"><td colspan="7">Нет файлов — выберите .doc / .docx / .pdf</td></tr>';
lucide.createIcons(); lucide.createIcons();
return; return;
} }
@@ -208,64 +207,83 @@ window.toggleClassifyDetail = async function(i) {
}; };
/** /**
* uploadFile(file, onProgress) — XHR-загрузка одного файла на бэкенд. * convertDoc(file, onProgress) — .doc → .docx через внешний libreoffice-сервис.
* С честным счётчиком времени.
*/
function convertDoc(file, onProgress) {
var startTime = Date.now();
if (onProgress) onProgress({ kind: 'converting', elapsed: 0 });
var fd = new FormData();
fd.append('files', file, file.name);
var timer = setInterval(function() {
var elapsed = Math.floor((Date.now() - startTime) / 1000);
if (onProgress) onProgress({ kind: 'converting', elapsed: elapsed });
}, 1000);
return fetch(CONVERT_URL + '?_=' + Date.now(), { method: 'POST', body: fd })
.then(function(r) {
if (!r.ok) throw new Error('Конвертация: HTTP ' + r.status);
return r.blob();
})
.then(function(blob) {
clearInterval(timer);
if (blob.size === 0) throw new Error('Конвертация: пустой результат');
return new File([blob], file.name.replace(/\.doc$/i, '.docx'), {
type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
lastModified: Date.now()
});
})
.catch(function(e) {
clearInterval(timer);
throw e;
});
}
/**
* ⛔ НЕ МЕНЯТЬ ⛔ uploadFile — fetch-загрузка с честным счётчиком времени.
* *
* Особенности: * v2.0.3: вместо фейкового ↑N% — честный счётчик ⏳ соединение... Nс → ⏳ отправка... Nс.
* - .doc (не .docx!) конвертируется через CONVERT_URL перед загрузкой * fetch() не даёт реальный upload progress, поэтому считаем секунды.
* - onProgress(pct) — callback с процентом загрузки (0-100) * Кэш-бастинг: ?_=Date.now()
* - Возвращает Promise<ответ API> с полями doc_id, contract_id, parsed
* - Таймаут 180с (большие PDF)
*/ */
function uploadFile(file, onProgress, zipSource) { function uploadFile(file, onProgress, zipSource) {
return new Promise(function(resolve, reject) { var startTime = Date.now();
// .doc → конвертация в docx (старый формат Word) var phase = 'connecting'; // connecting → uploading
var isDoc = file.name.toLowerCase().endsWith('.doc') && !file.name.toLowerCase().endsWith('.docx'); if (onProgress) onProgress({ kind: 'connecting', elapsed: 0 });
var uploadFile = file; var fd = new FormData();
var uploadName = file.name; fd.append('files', file, file.name);
if (state.contractId) fd.append('contract_id', state.contractId);
fd.append('batch_id', state.batchId);
if (zipSource) fd.append('zip_source', zipSource);
function doUpload() { // Честный счётчик: каждую секунду обновляем elapsed
var xhr = new XMLHttpRequest(); var timer = setInterval(function() {
var fd = new FormData(); var elapsed = Math.floor((Date.now() - startTime) / 1000);
fd.append('files', uploadFile, uploadName); if (onProgress) onProgress({ kind: phase, elapsed: elapsed });
if (state.contractId) fd.append('contract_id', state.contractId); }, 1000);
fd.append('batch_id', state.batchId);
if (zipSource) fd.append('zip_source', zipSource);
xhr.open('POST', UPLOAD_URL);
xhr.upload.onprogress = function(e) {
if (e.lengthComputable && onProgress) onProgress(Math.round(e.loaded / e.total * 100));
};
xhr.onload = function() {
try {
var r = JSON.parse(xhr.responseText);
if (r.ok) resolve(r);
else reject(new Error(r.error || 'Неизвестная ошибка'));
} catch(e) { reject(new Error('Некорректный ответ')); }
};
xhr.onerror = function() { reject(new Error('Сеть')); };
xhr.ontimeout = function() { reject(new Error('Таймаут')); };
xhr.timeout = 180000;
xhr.send(fd);
}
if (isDoc) { return fetch(UPLOAD_URL + '?_=' + Date.now(), { method: 'POST', body: fd })
var xhr = new XMLHttpRequest(); .then(function(r) {
xhr.open('POST', CONVERT_URL); if (!r.ok) throw new Error('HTTP ' + r.status);
xhr.responseType = 'blob'; return r.json();
xhr.onload = function() { })
if (xhr.status === 200 && xhr.response.size > 100) { .then(function(data) {
uploadFile = xhr.response; clearInterval(timer);
uploadName = file.name.replace(/\.doc$/i, '.docx'); if (data.ok) {
doUpload(); var elapsed = Math.floor((Date.now() - startTime) / 1000);
} else { if (onProgress) onProgress({ kind: 'uploading', elapsed: elapsed });
reject(new Error('Конвертация .doc')); return data;
} }
}; throw new Error(data.error || 'Неизвестная ошибка');
xhr.onerror = function() { reject(new Error('Конвертер')); }; })
xhr.send(file); .catch(function(e) {
} else { clearInterval(timer);
doUpload(); if (e.message === 'Failed to fetch' || e.name === 'TypeError') {
} throw new Error('Сеть');
}); }
throw e;
});
} }
/** /**
@@ -425,7 +443,7 @@ async function addZipFile(file) {
*/ */
async function addRegularFile(file, zipSource) { async function addRegularFile(file, zipSource) {
// Создать запись с начальным статусом // Создать запись с начальным статусом
var entry = { name: file.name, lastModified: file.lastModified, size: file.size, file: file, status: { kind: 'uploading', pct: 0 }, zip_source: zipSource || null }; var entry = { name: file.name, lastModified: file.lastModified, size: file.size, file: file, status: { kind: 'connecting', elapsed: 0 }, zip_source: zipSource || null };
// ДЕДУПЛИКАЦИЯ: ключ = (zip_source, name), чтобы одноимённые файлы из разных ZIP не затирались // ДЕДУПЛИКАЦИЯ: ключ = (zip_source, name), чтобы одноимённые файлы из разных ZIP не затирались
var dupKey = (zipSource || '') + '/' + file.name; var dupKey = (zipSource || '') + '/' + file.name;
@@ -449,9 +467,18 @@ async function addRegularFile(file, zipSource) {
render(state); render(state);
try { try {
// XHR-загрузка с прогрессом // .doc → конвертация через libreoffice-сервис
var resp = await uploadFile(file, function(pct) { var uploadTarget = file;
state.files[rowIdx].status = { kind: 'uploading', pct: pct }; if (file.name.toLowerCase().endsWith('.doc')) {
uploadTarget = await convertDoc(file, function(st) {
state.files[rowIdx].status = st;
render(state);
});
state.files[rowIdx].name = uploadTarget.name;
}
// fetch-загрузка с честным счётчиком времени
var resp = await uploadFile(uploadTarget, function(st) {
state.files[rowIdx].status = st;
render(state); render(state);
}, zipSource); }, zipSource);
// Бэкенд возвращает doc_id и contract_id (нижний регистр — Python keys) // Бэкенд возвращает doc_id и contract_id (нижний регистр — Python keys)
@@ -511,36 +538,62 @@ async function finalizeUpload() {
} }
/** /**
* onFilesSelected(newFiles) — Оркестратор загрузки (Фаза 1). * ⛔ НЕ МЕНЯТЬ ⛔ onFilesSelected — все строки сразу, потом загрузка.
* *
* ПАТТЕРН: * v2.0.2: Фаза 1 — все строки в таблицу. Фаза 2 — загрузка по одной.
* 1. Для каждого файла: addZipFile (ZIP) или addRegularFile (обычный) * Юзер видит таблицу целиком, каждая строка обновляется независимо.
* 2. finalizeUpload — завершить цикл
*
* НЕ удаляет существующие файлы — только добавляет новые.
* Дубликаты обрабатываются через confirm() в addRegularFile.
* Удаление — только вручную (кнопка ✕).
*
* Вызывается из fileInput.addEventListener('change', ...).
*/ */
async function onFilesSelected(newFiles) { async function onFilesSelected(newFiles) {
if (newFiles.length === 0) return; if (newFiles.length === 0) return;
fileInput.disabled = true; fileInput.disabled = true;
// Сбросить прогресс пайплайна при добавлении новых файлов
resetStepper('stepUpload'); resetStepper('stepUpload');
// Обработать каждый файл // Фаза 1: ВСЕ строки в таблицу сразу
for (var i = 0; i < newFiles.length; i++) { for (var i = 0; i < newFiles.length; i++) {
var f = newFiles[i]; var f = newFiles[i];
if (f.name.toLowerCase().endsWith('.zip')) { if (f.name.toLowerCase().endsWith('.zip')) {
await addZipFile(f); await addZipFile(f);
} else { continue;
await addRegularFile(f);
} }
state.files.push({ name: f.name, lastModified: f.lastModified, size: f.size, file: f, status: { kind: 'connecting', elapsed: 0 }, zip_source: null });
state.files[state.files.length - 1]._pendingFile = f;
}
render(state);
// Фаза 2: загрузка по одному с обновлением строки
for (var j = 0; j < state.files.length; j++) {
var entry = state.files[j];
var pendingFile = entry._pendingFile;
if (!pendingFile) continue;
delete entry._pendingFile;
var f = pendingFile;
try {
// .doc → конвертация через libreoffice-сервис
var uploadTarget = f;
if (f.name.toLowerCase().endsWith('.doc')) {
uploadTarget = await convertDoc(f, function(st) {
entry.status = st;
render(state);
});
entry.name = uploadTarget.name;
}
var resp = await uploadFile(uploadTarget, function(st) {
entry.status = st;
render(state);
});
if (resp && resp.contract_id) state.contractId = resp.contract_id;
entry.doc_id = resp.doc_id;
entry.status = { kind: 'uploaded' };
entry.uploaded = true;
var pr = resp.parsed;
var elapsed = pr && pr.status === 'parsed' ? '0.0' : null;
applyParseResult(entry, pr, elapsed);
} catch(err) {
entry.status = { kind: 'error', text: err.message };
}
render(state);
} }
// Завершить цикл
await finalizeUpload(); await finalizeUpload();
} }
+8 -8
View File
@@ -73,7 +73,7 @@
<body> <body>
<div class="topbar"> <div class="topbar">
<img src="/static/logo.svg" alt="Nubes"> <img src="/static/logo.svg" alt="Nubes">
<span class="title">Сверка договоров — LLM AI-driven Event Sourcing <span style="font-weight:400;color:var(--muted);font-size:12px;">v2.0.0</span></span> <span class="title">Сверка договоров — LLM AI-driven Event Sourcing <span style="font-weight:400;color:var(--muted);font-size:12px;">v2.0.9</span></span>
<div id="pipelineStepper" style="display:flex;gap:8px;font-size:11px;align-items:center;color:var(--muted);"> <div id="pipelineStepper" style="display:flex;gap:8px;font-size:11px;align-items:center;color:var(--muted);">
<span id="stepUpload">○ Загрузка</span><span></span> <span id="stepUpload">○ Загрузка</span><span></span>
<span id="stepClassify">○ Классификация</span><span></span> <span id="stepClassify">○ Классификация</span><span></span>
@@ -90,7 +90,7 @@
Загрузка договоров/приложений/спецификаций Загрузка договоров/приложений/спецификаций
</div> </div>
<div class="card-body"> <div class="card-body">
<input type="file" id="fileInput" accept=".docx,.doc,.pdf,.zip" multiple style="margin-bottom:6px;width:100%;"> <input type="file" id="fileInput" accept=".doc,.docx,.pdf,.zip" multiple style="margin-bottom:6px;width:100%;">
<div style="text-align:right;font-size:11px;color:var(--muted);margin-bottom:6px;">Порядок определяется автоматически при классификации</div> <div style="text-align:right;font-size:11px;color:var(--muted);margin-bottom:6px;">Порядок определяется автоматически при классификации</div>
<div style="font-size:11px;color:var(--muted);margin-bottom:6px;">⚠ При совпадении имён — запрос на перезапись (OK / Отмена). Файлы из ZIP-архивов загружаются через тот же поток.</div> <div style="font-size:11px;color:var(--muted);margin-bottom:6px;">⚠ При совпадении имён — запрос на перезапись (OK / Отмена). Файлы из ZIP-архивов загружаются через тот же поток.</div>
@@ -216,11 +216,11 @@
</div> </div>
</div> </div>
<script src="/static/state.js?v=1.0.179-flask"></script> <script src="/static/state.js?v=2.0.8"></script>
<script src="/static/app_utils.js?v=1.0.179-flask"></script> <script src="/static/app_utils.js?v=2.0.8"></script>
<script src="/static/files.js?v=1.0.179-flask"></script> <script src="/static/files.js?v=2.0.8"></script>
<script src="/static/groups.js?v=1.0.179-flask"></script> <script src="/static/groups.js?v=2.0.8"></script>
<script src="/static/compare.js?v=1.0.179-flask"></script> <script src="/static/compare.js?v=2.0.8"></script>
<script src="/static/app.js?v=1.0.179-flask"></script> <script src="/static/app.js?v=2.0.8"></script>
</body> </body>
</html> </html>