B²LAB

Why the data foundation comes first

Before you can build an AI model, you need data you can trust. When every machine uses a different format, collection breaks off partway, and the same field goes by different names, no model placed on top will produce results worth believing.

B²LAB is a big data optimization platform supporting collection, storage and processing, and monitoring of the data an organization already holds. It links the structured, unstructured and semi-structured data generated across the enterprise to secure stable raw data.

Modular structure

B²LAB supports data collection, storage and processing, and monitoring, with each system provided as a module. The aim is to raise how much of the organization’s data can actually be put to use.

Because you can start with only the modules you need and expand later, small and mid-sized manufacturers that cannot replace everything at once still have a path in.

Processing stages

The platform runs in four stages.

1. Collection and integration — It starts with a data owner declaring the entity and relationship schema as a YAML ontology. Database tables in QMS, EDMS and ERP arrive through change data capture and scheduled replication; CSV, JSON, PDF and Office files are read from file server paths. On the equipment side, heterogeneous protocols such as PLC, Modbus and RS-232/485 are normalized to OPC-UA and streamed through Kafka. Batch and real-time paths both land in the lakehouse.

2. Quality verification — Total and distinct counts, minimum and maximum values, null counts and format and type distributions are profiled automatically. On top of that, gaps, duplicates, out-of-range values, outliers and type consistency are adjudicated and written to the catalog, and a data curator reviews the findings and acts. Skip this stage and the graph built later cannot be trusted.

3. Mapping and linking — This is where the problem of each system calling the same product by a different code gets solved. Embeddings propose candidates that are likely to be the same entity, and an LLM adjudicates, promoting only those above 95% confidence to a single entity. Codes link across systems without hard-coded rules, and the promoted relationships form the record–product–material–supplier graph and the document version history index. Finally, an automated test confirms the ontology schema and the graph still agree.

4. Synchronization and delivery — Changes are applied and overall consistency is re-checked. The integrated data and knowledge graph are served to AI agents over REST APIs, while business systems receive real-time data through message subscription. For analysis, processed data goes to a service database and the raw data stays open to ad-hoc queries.

Adoption requirements

Deployment and network — Choose a dedicated customer cloud (VPC) or on-premise; air-gapped sites are served on-premise. Every component deploys as individual Docker containers on customer servers, and the batch and real-time (Kafka) collection paths can run on separate servers and containers. Firewall rules must allow the data lake server to reach source databases and file servers by IP and port; SFTP is recommended for file connections over an external network.

Sources it connects to — Databases: MSSQL, Oracle, PostgreSQL, MariaDB, IBM DB2 and Machbase. File servers over FTP, SFTP and SMB. Databases support scheduled table replication, view and SQL queries, and ad-hoc joins across heterogeneous systems. Equipment data lands through the API-based real-time (Kafka) path, and external systems can also push data directly.

Integration permissions — Database access to source systems such as QMS and EDMS is required. Collected data lands in the lakehouse, then synchronizes nightly by delta with a full consistency re-check weekly. The resulting knowledge graph is exposed through REST APIs and a web dashboard and console; a graph database (Neo4j) is an optional connection.

Permissions and operation — User and menu permissions and API keys are managed from an admin screen, and source integration accounts run with read-only rights. A data owner declares and reviews the YAML ontology. Collection pipeline logs and service API call logs are retained, and failed real-time loads are handled by automatic retry and ingestion throttling to minimize loss.

In production

In pharmaceutical and bio manufacturing, QMS and EDMS data is collected and synchronized nightly, with ontology-based entity mapping linking products, materials and contract manufacturers as single entities.

146,919 quality records were mapped into an entity and relationship graph, and a version history index was built over more than 63,000 electronic documents. Over 3,000 LLM-adjudicated entity mappings now make cross-system product and material lookup possible. ConsensBot runs on top of this foundation at the same customer.