DATAtokeniz turns any file into machine-learning-ready data
Upload a PDF, document, spreadsheet, web page, image, audio or video file. DATAtokeniz converts it into a structured dataset you can use to train, fine-tune, evaluate or ground an AI model — with every row traceable to the exact page, timestamp or character range it came from.
Sign in with Google to save finished datasets to your own Drive — DATAtokeniz only ever sees files it creates there.
PDF → graded eval setno key needed
Transformer
Input → guided handling → JSONL outputInput
Source material
Bring it from
Processed in the request and never stored. Audio and video are transcribed into timestamped segments; a public Kaggle link is downloaded and every row cites its file.
Declare source license— optional
Demo mode · nothing is saved. Sign in to raise the limits and keep a run history.
Transformer
Tell DATAtokeniz how to handle it
Schema architect
Designs the structure — never writes your rows
Add a source on the left, then tell me exactly how to turn it into a dataset.
Shapes I can architect
- Eval · grounded Q&ATest whether a model is right
- RAG · retrieval chunksGive a model your documents to search
- SFT · instruction pairsTeach a model your material
- Multimodal · frame + textPair what is seen with what is said
- extract
- segment
- enrich
- quality
- compile
Extracted only — the guide never writes dataset rows.
Output
ML-ready dataset
Your dataset lands here
- JSONL previewthe first rows, before you download anything
- Quality scoreduplicates, PII and weak grounding, counted
- Provenance recordevery row traced to its exact source location
Enabled the moment a run finishes.
About DATAtokeniz
DATAtokeniz is a data preparation tool operated by Numen Labs, LLC. It converts files you already have — PDFs, Word and PowerPoint documents, spreadsheets, web pages, images, audio and video — into machine-learning-ready datasets: structured, validated training data you can use to fine-tune a model, build a retrieval index, or run an evaluation.
Our mission
Make training data whose origin you can prove. Every row DATAtokeniz produces carries a reference back to the exact source file, page, timestamp or character range it came from. Text a model generated is labelled as generated, and is never silently mixed with text extracted from your documents.
What you get back
A JSONL dataset in the shape you asked for — evaluation, retrieval, instruction-tuning or multimodal — plus a quality report that counts duplicates, personal data and weakly grounded rows, and a data card recording how the set was built. The schema is public and versioned.
Why DATAtokeniz asks for Google
Google sign-in creates your account, and the drive.file scope lets DATAtokeniz save finished datasets to your own Google Drive. That scope grants access only to files you pick or that DATAtokeniz creates — the rest of your Drive stays invisible to us. Source files are processed in the request that carries them and are never written to DATAtokeniz storage.
Common questions
- How do I turn a PDF into training data for an AI model?
- Upload the PDF, describe what you want in plain language — for example "build an evaluation set with questions and reference answers, citing pages" — and DATAtokeniz extracts the text, splits it into segments, and compiles a JSONL dataset. Each row records the page it came from, so any answer can be checked against the source. No API key is needed to try it and nothing is stored.
- What file formats can DATAtokeniz convert into machine-learning-ready data?
- PDF, Word (DOCX), PowerPoint (PPTX), CSV and TSV spreadsheets, JSON and JSONL, HTML and live URLs, PNG and JPEG images, and audio and video files. Pasted text works too. Audio and video are transcribed from the audio track; images can be described or read by a vision model when one is configured.
- What is a JSONL dataset, and which format does DATAtokeniz produce?
- JSONL is JSON Lines: one complete JSON object per line, which is the format most fine-tuning and evaluation pipelines read. DATAtokeniz produces four shapes — evaluation (question, reference answer, citation), retrieval or RAG (passage with citation), instruction tuning or SFT (instruction and response, also available as ShareGPT and Alpaca), and multimodal. It also exports canonical JSON, CSV, Parquet and a Markdown data card.
- How is this different from just asking ChatGPT to make a dataset?
- A model asked to produce a dataset invents rows that read plausibly and cite nothing. DATAtokeniz extracts from your actual source first, and every row keeps a reference to the file, page, timestamp or character range it came from. Where a model does contribute, that text is labelled as generated and kept separate from extracted text rather than blended into it. The difference matters the moment anyone asks where a training example came from.
- Does DATAtokeniz need an API key?
- No. The demo runs the full deterministic pipeline with no account and no key, and stores nothing. An AI Gateway key adds model-assisted enrichment, and a Whisper-compatible key adds audio and video transcription. When a capability is unavailable the run emits a warning and continues rather than failing or pretending.
- How does data provenance work, and why does it matter for the EU AI Act?
- Every segment traces to a source and an exact location within it, and every field records whether it was extracted, computed, model-generated, or human-verified. From that, DATAtokeniz can build an EU AI Act Article 10 data-governance record: origin with per-source checksum and licence, preparation steps, assumptions and limitations, and personal-data findings. Article 10 applies to Annex III high-risk systems from 2 December 2027. DATAtokeniz produces the evidence; it does not make anyone compliant.
- Is my data used to train models, or stored on your servers?
- No and no. Source files are processed in the request that carries them and are never written to DATAtokeniz storage. Finished datasets go to your own Google Drive or a browser download. Google access uses the drive.file scope, which reaches only files you pick or that DATAtokeniz creates. Video face-landmark extraction runs entirely in your browser, so the video never leaves your machine.