Core image analysis system

Parallel scanning and data persistence

Separated physical equipment use from long-running backend processing. Once acquisition finishes, the scanner becomes available for the next sample while the previous result continues to be persisted independently.

Problem

The scanner remained occupied for 4–5 minutes, although acquisition took about one minute. The next sample had to wait for persistence.

My responsibility

Designed contracts and implemented backend orchestration, workflow states, intermediate result capture, safe retries, and recovery scenarios.

Result

The scanner is released after the result is captured. The next scan and persistence of the previous result proceed independently.

4–5 min

scanner occupancy per metre before the change

≈ 1 min

for acquisition, after which the equipment is released

3–4 min

persistence continues independently from the next scan

What changed for the user

The equipment acquires core images and measurements from multiple sensors. Previously, equipment availability depended on completion of the entire backend workflow. Users now see separate progress for acquisition and persistence.

Project task list showing a new scan and persistence of the previous result running at the same time — Separate task states: scan 15 is in progress while the result of scan 14 is being persisted independently.

Separating the workflows

After acquisition, the backend captures a result snapshot and releases the equipment. Two independent branches follow: scanning the next sample and persisting the previous result.

Before · scanner occupied for 4–5 minutes

AcquisitionAbout 1 minute

Processing and persistenceAnother 3–4 minutes; scanner unavailable

Next sample

After

Acquisition

Capture result snapshot

Release scanner

Equipment branchScan the next sample

Backend branchPersist the previous result · about 3–4 minutes

Dependency diagram, not a timeline. The branches after equipment release are independent; total processing time and overall throughput were not measured separately.

Parallel execution did not require a major rewrite of the existing pipeline. The key changes were event handling and separating task states.

Reliability and my contribution

I implemented backend orchestration and contributed to contracts between the main backend, scanner service, and analysis service. Other developers worked on the equipment mechanics and parts of the scanner service.

Capture before release. An intermediate result snapshot allows processing to continue without querying equipment that has already been released.
Short transactions. States and results are persisted in separate operations. File, network, and long-running compute operations do not hold a database transaction.
Connection recovery. The backend reconciles its stored state with the scanner service to determine how the workflow can continue safely.
Automated checks. Tests cover transaction boundaries, repeated processing, and partial failures.

I also contributed to aligning data from multiple sensors and implemented separate progress phases.

Result and measurement limits

Scanner occupancy per metre fell from approximately 4–5 minutes to about one minute. Persistence, taking another 3–4 minutes, no longer blocks the next sample.

This does not mean a fourfold reduction in total processing time. The confirmed effect is earlier equipment availability.

Stack: Java, Spring Boot, Spring Modulith application events, PostgreSQL, WebSocket, REST, file storage, JUnit, Mockito, integration testing.