A condensed field manual for the Data & More platform: the conceptual model, the end-to-end data pipeline, how OCR turns raw documents into clean text and signals, how the Profiler classifies what it finds, the AI Profiler that powers the heavy lifting, and the technology stack underneath.
Seven sections. Each one the smallest amount of detail that still makes sense.
The platform runs continuously, not on a fixed quarterly cadence. The process has a name, Classify, Verify, Delete: the platform classifies what sits in the archive, the data owner verifies the findings on their own ground, and the result is deletion (or archive, edit or restrict) carried out in the source system. Sources and exceptions are the one-time configuration that feeds the loop.
Classify, the platform's jobVerify, the data owner's jobDelete, the continuous goal
Where the data comes from
Connected sources
Each source is wired through one of the platform's ingestion connectors. The cycle starts here and writes findings back here too.
Deletion is never automatic. The data owner reviews each finding locally and picks one of three actions, recorded in the audit log.
Edit, the document is corrected, redacted or annotated in the source system.
Archive (retention), the record is moved to an archive with a defined retention rule.
Restrict, the record stays put but is flagged for restricted access (private data handling).
Section 02Conceptual model
The conceptual model
Every request enters through one TLS-terminating NGINX proxy and is routed to the application tier: the Vue client, the main Flask API, and the IAM, analytics and LLM services. The API hands slow work to an asynchronous backbone of Celery workers over RabbitMQ, which also carries the scan, ingest and enforce events for the Java tier. The heavy lifting (crawling sources, extracting text, enforcing policy) runs in that Java tier. Underneath it all sits the data layer, with Elasticsearch as the document store every service shares.
Request and data pathPersisted store / outputSub-component inside a tier
Section 03Data pipeline
From a raw source to an enforced policy
A document makes the same journey every time. A source is configured once; from then on the platform crawls it, extracts its text, profiles it for personal data, checks it against the tenant's policies, acts on the verdict, and reports the result. Cheap, deterministic stages run first; the expensive AI and enforcement steps run only on what reaches them.
Main pipeline pathShared store, touched at every stageOutput to the user
Section 04OCR
The OCR pipeline
Pages move left to right. The MRZ pass only fires when the ID-or-not detector says a page is an identity document; everything else continues to the rich OCR pass. Each stage emits annotations the next stage can use as hints.
Main data pathConditional branch, ID onlyVision signalDocument annotation
OCRRationale
Why layer the work at all
A real document corpus is heterogeneous: born-digital PDFs, scanned letters with a coffee stain, multilingual contracts, and passports with a machine-readable zone all land in the same inbox. The pipeline routes each page through cheap, deterministic passes first and reserves expensive deep-learning OCR for the cases that earn it. Vision signals ride alongside the text, so a consumer can ask is this contract signed or is this upload actually a passport without re-reading the file.
Pass 01PDF reader
The front door
Two PDFs that look identical to a person can be wildly different inside: one a born-digital export with a perfect text layer, the other a phone photo flattened to PDF at 72 DPI and rotated four degrees. The reader normalises that asymmetry so downstream OCR never has to.
What it does
Rasterises each page to a standard 300 DPI so engines see consistent character sizes.
Extracts the embedded text layer when present, short-circuiting OCR entirely for born-digital files.
Deskews, dewarps and fixes orientation, then crops scanner-bed borders.
Splits multi-page documents into a per-page stream the rest of the pipeline handles independently.
Unique advantage
Free text wins. A born-digital PDF skips OCR, instant, perfect, error-free.
Consistent input. Every later model can assume an upright, sane-DPI image.
One source of truth. The image rendered here is reused by every later worker, no double rasterisation.
Pass 02Tesseract
The fast baseline
Tesseract is the workhorse: fast, CPU-friendly, 100+ languages, and structured output, word boxes, line boxes, per-word confidence. For the long tail of clean office documents it solves the problem outright.
What it does
Runs the LSTM recogniser over each page and emits hOCR / TSV with words, lines and boxes.
Attaches a per-word confidence score that decides where EasyOCR needs a second pass.
Returns reading order and layout, so paragraphs reconstruct cleanly.
Unique advantage
Speed and cost. Pure CPU, no GPU dependency, scales horizontally, stays cheap.
Deterministic. Same input, same output: easy to test, cache and diff in CI.
Layout-aware. Word boxes and reading order are first-class outputs.
Confidence is a routing signal. Low-confidence regions become EasyOCR's input.
Tesseract is deliberately the first OCR pass, good enough for most pages. The expensive passes only run where it falls short.
Pass 03Combined worker
Signatures, faces, and one decision
The combined worker is the vision lane. Rather than running three jobs that each reload the page image, allocate a tensor and warm a model, it runs three detectors over the same in-memory image in one pass, producing document-level signals that text alone can't answer.
Detector A
Signature
Most contracts only matter once countersigned. A small CNN scans for ink-like strokes that read as a handwritten signature and returns boxes plus a score.
Turns is this contract executed into a boolean.
Box plus page index lets a UI jump to the signed line.
The nearest text line, the printed name, can be paired to the box.
Detector B
YuNet
YuNet is a tiny face detector built to run in real time on commodity hardware. Here it isn't about faces, it's about portraits as a document feature.
A face in the corner is a strong cue for an ID, passport or licence.
About 1 ms per crop on CPU, cheap enough to run on every page.
Presence of a face can drive redaction or special handling.
Detector C
ID or not
A binary classifier that reads the whole page and answers one question: is this an identity document? Its job is to decide which pages reach the MRZ specialist.
Avoids running MRZ on every page, most have none.
Uses YuNet's face hit and Tesseract's text as features.
A small head over a small backbone: fast, easy to calibrate.
Why combined? Loading the image, colour-converting, resizing and warming a model account for most of any detector's latency. Running all three over one shared image, one process, one tight loop, collapses that overhead and keeps the annotations self-consistent, because all three saw the same pixels.
Pass 04MRZ
The passport specialist
The machine-readable zone at the foot of a passport or ID card is a strict, ICAO-9303 format: a fixed character set (A–Z 0–9 <), fixed positions, and built-in check digits. A generic OCR will read it but garble O/0 or 1/I. This pass is purpose-built for the strip.
What it does
Runs only when the ID-or-not classifier flags the page, so it never wastes work.
Uses an MRZ-tuned recogniser and grammar that only emit valid MRZ characters.
Parses the fields: type, issuing country, surname, given names, document number, nationality, date of birth, sex, expiry.
Verifies check digits, a forged or mis-read field is caught by arithmetic, not by guessing.
Unique advantage
Structured, not free text. KYC code consumes a typed record, not a string.
Self-validating. Check digits give a guarantee generic OCR can't match.
Narrow domain, high accuracy. Small character set, fixed layout, a specialist wins by a wide margin.
Pass 05EasyOCR
The deep-learning last mile
EasyOCR is a deep-learning OCR, a CRAFT text detector plus a CRNN recogniser on PyTorch. Heavier than Tesseract, but it shines exactly where Tesseract struggles: low-contrast or stylized fonts, curved or rotated text, photos of receipts, and many non-Latin scripts. Here it is the fallback and the rich-OCR pass.
What it does
Re-reads the regions where Tesseract returned low confidence, surgical, not whole-page.
Handles non-Latin scripts a given Tesseract deployment isn't configured for.
Provides a second opinion: agreement between the two engines raises confidence sharply.
Unique advantage
Robustness. CNN recognition copes with photos, perspective and unusual fonts.
Coverage. 80+ languages out of the box.
Targeted. Only runs where Tesseract wasn't confident, so the slow path stays small.
Cheap first, expensive only where needed, that is what keeps the pipeline fast on average without losing the long tail.
OCRRouting
The decision a page makes
OCRAt a glance
What each pass adds
Pass
Output
Cost
When it shines
PDF reader
normalized images, text layer
very low
born-digital PDFs
Tesseract
word boxes + confidence
low · CPU
clean office documents
Signature
signed flag + boxes
low
contracts, compliance
YuNet
face boxes, count
very low
spotting portraits / IDs
ID-or-not
isID, document type
low
routing to MRZ
MRZ
typed record + check digits
medium
passports, national IDs
EasyOCR
text + confidence
high · GPU-friendly
stylized, noisy, multilingual
OCRPrinciples
What the pipeline believes
Cheap first, expensive only when earned
Every pass exists because the one before it can't handle a specific failure mode. Tesseract carries the bulk; EasyOCR is invoked only for the words it couldn't read; MRZ only when the page is truly an ID.
Signals, not just text
OCR is necessary but rarely sufficient. The combined worker emits document-level signals, signed, portrait present, identity document, that the business logic downstream actually needs.
Specialists beat generalists in narrow domains
A passport MRZ is a tiny, strict format with check digits. A specialist parser is more accurate, and self-validating, in a way no generic OCR can be on the same strip.
Share the work
The combined worker exists because loading and preprocessing the image is the slow part. Three detectors over one in-memory image collapse that overhead and keep the annotations consistent.
Section 05Profiler with classification
The Profiler and its taxonomy
The Profile stage in the data pipeline runs in two parallel modes. Logic Profile (java_profiler) handles regex, language detection and rule-based entity extraction. AI Profile (ai-profiler, the spaCy NLP engine) handles NER, keyword phrase matching and dependency-grammar checks. Both write their findings into the same shared taxonomy below.
Findings are organised into top-level categories, each holding many entry types. Categories and entry types are driven by per-customer dictionaries loaded from Elasticsearch, so each subscription can enable, disable or extend any of them. The list below mirrors the Document Classes view in the admin UI.
Category 01
Privacy Classification
The bulk of GDPR-relevant detection: identifiers, special categories, and document classes whose presence alone is a signal a record needs governance.
PassportHealth infoCertificates / PermitTravel infoPolitical OrientationWork absencePower of attorneyInsurance infoLocationCriminal BehaviorCriminal RecordEmployee warningNational ID CardGrant ApplicationPayment CardReligious OrientationSexual orientationNational ID numberWillsEthnic originEmployment infoDrivers licenseUnion MembershipSalary / financial infoTax InfoHealth cardEmployee TerminationMisc. IDRecruitmentEducational info
A parallel taxonomy for content that endangers the organisation rather than a person: secrets, infrastructure, and security operations.
Passwords & SecretsSource CodeInfrastructure ConfigVulnerability assessmentLog FilesSecurity IncidentsCCTV camera locationsDigital certificatesSecurity requirementsNetwork access control
10 entry types · tagged with SECURITY org type
Category 03
QA
A workbench category used by the data team to stage and trial new entry types before they are promoted into one of the public taxonomies. Off by default for tenants.
Category 04
Tag cleanup
Maintenance category that holds retired entry types and merge targets, so historical findings remain interpretable while new scans use the current taxonomy.
Category 05
Special classifications
Tenant-specific categories for entry types that do not belong to a global taxonomy. A customer can extend this category with their own dictionaries.
Per entry type
What an entry type carries
Active, Report, Display, One-Time Tagging, Search Tag, Tag in Outlook: per-tenant toggles that control where the finding shows up.
# Documents: live count of records currently bearing the tag.
Status, Org Type, Token #: provenance and validation of the entry type itself.
Execution time and Tagging execution time: cost telemetry on how long the entry type takes to evaluate per document.
Tagging finished and Subscription updated: timestamps for the last full pass and the last dictionary sync.
Named entities
Personal identifiers (from NER)
Sitting alongside the dictionary-driven categories, the spaCy NER lens emits identifier findings that are not configurable per tenant.
PersonFull nameOrganisationLocationPlace / GPEDateTimeNationality / group
Section 06AI Profiler
The AI Profiler deep dive
The same detection core (handle_doc) powers a high-volume background pipeline and a live request endpoint. Batch work flows left to right through the scheduler, a bounded queue and a pool of spaCy workers; the realtime endpoint feeds a single piece of text straight into the core and returns JSON.
Batch data pathRealtime endpointWorker / resultDetection core
AI ProfilerThe job
Find the sensitive data, label it, hand it back
The Profiler reads documents already indexed in Elasticsearch. For each one it detects names, places and dates, then looks for language that reveals special categories of personal data: health, religion, politics, sexual orientation, ethnicity, criminal history, union membership and employment actions. Every match is tagged with a type, the matched text and its position, then saved back onto the document, so compliance work starts from facts rather than guesswork.
AI ProfilerTwo ways in
A pipeline and an endpoint
Mode A
Scheduled pipeline
A background thread continuously polls Elasticsearch for documents flagged DS_Status: REQUEST_AI and feeds them to a pool of worker processes. This is how bulk archives get profiled.
Mode B
REST endpoint
A POST /profile-text route profiles a single piece of text on demand, returning colour-coded matches as JSON. A /health route reports liveness.
AI ProfilerThe batch pipeline
From flagged document to finished profile
On startup the app waits for the Elasticsearch cluster to turn healthy, then launches the scheduler and the worker pool. The cycle below repeats forever.
01 Scheduler queries Elasticsearch. Every cycle it scans the data index for documents where DS_Status = REQUEST_AI, paging 50 at a time with search-after.
02 Each document becomes a task. Documents are wrapped as tasks and pushed onto a shared multiprocessing queue. If the queue fills, the scheduler pauses, providing natural back-pressure so memory stays bounded.
03 Workers pick up tasks. A pool of spaCy worker processes (2 by default) pulls tasks in parallel, runs the detection core and merges any pre-existing labels already on the document.
04 Detection core runs. The text is screened, language-routed and passed through up to three detection algorithms. This is handle_doc, expanded below.
05 Results written back. Findings are de-duplicated and bulk-updated onto the document, and the status is set to FINISHED. Even on error the status flips to FINISHED, so nothing is reprocessed endlessly.
Back-pressure on a 1000-deep queue keeps the whole pipeline within a fixed memory envelope, no matter how large the archive.
AI ProfilerInside the core
Screen, route, then match
Before any model runs, handle_doc filters out work it should not do, then picks the right tool for the language. Only after that do the matchers fire.
Step 1
Decide what to process
Only whitelisted document types are profiled (doc, docx, pdf, eml, txt and more). Structured formats like json, xml and log are skipped. Text longer than 20,000 characters is flagged and truncated, and unknown-language content is dropped.
Step 2
Pick the language strategy
If a language has grammatical dependency patterns, the whole text is analysed at once. Otherwise the text is split into sentences and each is analysed in turn, with the multilingual model as a universal fallback.
AI ProfilerDetection algorithms
Three lenses on the same text
Each finding carries a prefixed label so downstream systems know how it was found. The three lenses run over the same sentence and their results are merged.
01 · NER
Named entities
spaCy's statistical model picks out people, organisations, places, dates and times. Names of the right shape (two to three distinct alphabetic words, no digits) are promoted to full-name findings.
A lemma-aware phrase matcher scans for terms from per-customer dictionaries of sensitive vocabulary, so it catches inflected forms rather than only exact strings.
labels: S_K_<TYPE>
03 · Grammar
Dependency matching
Grammatical patterns confirm a real claim: a person or allowed pronoun is the subject, and a sensitive keyword is the object. This cuts false positives by demanding context, not just a word.
labels: S_S_<TYPE>
AI ProfilerReach
Built for scale and reach
15+language models
3detection methods
1ktask queue depth
9sensitive categories
Dedicated models ship for English, Danish, German, Dutch, French, Italian, Spanish, Swedish, Norwegian, Finnish, Polish, Portuguese, Lithuanian, Croatian (also serving Serbian, Bosnian and Montenegrin) and Ukrainian, with a multilingual model covering everything else and a universal sentence splitter underneath.
From a configured mailbox to an enforced retention rule, every document follows one path through a polyglot estate held together by a single shared store. The architecture is plural by design; the source of truth is not.
~42 services · Python + Java core · Elasticsearch at the centre