Atlas turns Brazilian public company data into trusted, business-ready datasets. It acquires monthly Receita Federal and official geography inputs, preserves immutable raw evidence, and builds quality-gated bronze, silver, and gold data products for company profiles, relationships, and lead generation.
Atlas is currently a local Scala and Spark data platform. A separate indexer foundation can fail-closed validate contracted gold bundle inputs, but no serving database or supported query service exists yet. Serving projections, an API, and an application remain roadmap work.
flowchart LR
A["Receita and IBGE<br/>public inputs"] --> B["Raw<br/>immutable evidence"]
B --> C["Bronze<br/>source-faithful Parquet"]
C --> D["Silver<br/>normalized entities"]
D --> E["Gold<br/>business-ready products"]
E -. "roadmap" .-> F["Serving, API,<br/>and application"]
Atlas requires JDK 17, sbt 1.10 or newer, Python 3, and enough local disk for source archives, Parquet, and Spark working data.
./atlas compile
./atlas test
./atlas download receita company-data --release 2026-07
./atlas refresh receita company-data --release 2026-07
./atlas releases validate-bundle --full
./atlas statusDownloads use the network and write immutable local raw inputs. Refresh is local-only and publishes an atomic bundle only after all required components and quality gates succeed. Generated data and credentials must never be committed.
- Build and test Atlas
- CLI command reference
- Architecture
- Datasets and important fields
- Operate the company-data pipeline
- Query Atlas with DuckDB
- Product objective and roadmap
- Complete documentation index
apps/etl/— Scala and Spark ingestion, transformation, quality, publication, and export codeapps/indexer/— independent serving contracts and validated gold-bundle reader; projection work is plannedapps/etl/data/— ignored local raw and generated dataapps/etl/examples/duckdb/— executable local inspection and product-query examplesdocs/manual/— supported user-visible behaviordocs/operations/— commands, runbooks, recovery, and troubleshootingdocs/specs/— exact dataset, schema, quality, and operational contractsdocs/atlas_unified_plan.md— product direction, MVP objective, and roadmap sequencing
Raw, bronze, silver, gold, exports, and serving projections have separate contracts. Product surfaces must consume contracted gold data or rebuildable projections of it, never raw files or ad hoc silver semantics.