Chapter 304: Dust in the Archives

The server room’s AC vent kept exhaling a steady stream of cold air. On the monitor, the log scroll bar had been running for four hours. Lin Chen leaned back in his ergonomic chair, his left foot resting on a custom hardwood footpad, a worn copy of Computer Networks propped under his knee. The spine was faded to white, the corners curled; he’d bought it for two yuan when the university library was discarding old stock.

The pipeline status bar read: Processed: 142,000. Exception Queue: 3,841. Progress: 18.7%.

He lifted the enamel mug from the desk. The goji berry water inside had gone completely cold. He took a sip. The astringent taste slid down his throat, triggering a faint spasm in his stomach. He hadn’t added sugar, nor had he changed the tea leaves. Frugality was carved into his bones. Even though the company’s accounts could now cover six months of cash flow, he still used this chipped, paint-peeling mug.

His phone screen lit up. A message from Su Man: “The IT department at Provincial Second Hospital dropped off the hard drive. Three years of outpatient logs, the raw pre-anonymization database. Mixed formats: CSV exports from pre-2018 systems, JSON from post-2019, with about 15% PDF scans mixed in. They only gave us seven days. The drive arrives at 3 PM.”

Lin Chen replied: “Received. Preparing backup servers. I’ll take it this afternoon.”

He closed the chat window and pulled up a sample of the exception queue logs. The first 3,841 errors clustered around three issues: first, inconsistent timestamp formats—some included timezone offsets, others were just YYYYMMDD; second, the diagnosis fields were riddled with garbled text from OCR conversions of handwritten notes, with GB2312 and GBK encodings mixed together, and some rare characters replaced by ?; third, patient IDs didn’t match visit serial numbers, revealing a clear fracture from historical system migrations.

This wasn’t dirty data. It was geological strata.

Lin Chen opened his error notebook and wrote on a fresh page: Multi-source heterogeneity. Encoding fault lines. ID mapping fracture. The pen tip scratched lightly against the paper. He closed the notebook and began refactoring the parsing layer of DataAnonymizerV1.

At 2:50 PM, Su Man pushed open the server room door, carrying a black shockproof hard drive case. She didn’t speak, just set it on the console and closed the door behind her. The room settled back into the low-frequency hum of server fans.

Lin Chen connected the drive and mounted it to an isolated sandbox. Total data volume: 47.6 GB. He took a deep breath and hit the start command.

The progress bar began to climb. The first ten minutes were smooth. At the fourteen-minute mark, the log window suddenly flashed a string of red text: PARSE_ERROR: Invalid UTF-8 sequence at offset 0x8F2A. The pipeline paused.

Lin Chen didn’t frown. He had anticipated this. He opened a hex viewer, navigated to the error offset, and examined the raw bytes: 0xD2 0xBB. In GBK, it was the character “一” (one), but it crashed the UTF-8 decoder outright. He wrote a pre-check script to scan the entire database’s byte distribution and calculate the proportion of non-UTF-8 characters. The result: 12.4%.

He created a new EncodingFallback module. When it encountered an illegal byte sequence, it wouldn’t halt or discard the record. Instead, it would tag the original encoding, attempt secondary identification using the chardet library, and if that failed, route it to a quarantine zone while preserving a raw hexadecimal snapshot. The script compiled successfully, and the pipeline restarted.

The progress bar resumed its crawl. It was about 30% slower than before, but it no longer crashed.

Lin Chen stood up and went to the restroom to splash cold water on his face. The man in the mirror had sunken eye sockets and a bluish shadow of stubble. He looked down at his left foot; the skin around his ankle had gone pale from prolonged pressure. Gripping the edge of the sink, he slowly performed two sets of stretches. His muscle fibers ached with a familiar, heavy soreness, like rusty gears being forced to mesh. He closed his eyes, counted to ten, and walked back to his desk.

By 5 PM, the processed count had broken 200,000. The exception queue had swelled to 11,000.

Su Man knocked and entered, setting down a takeout container and two bottles of mineral water. “Eat first. The IT department mentioned that between 2016 and 2017, the hospital switched HIS systems. The old data was manually exported, and a lot of fields are blank. Do you want to skip that portion?”

“Can’t skip.” Lin Chen opened the lunchbox. Simple green pepper and pork with rice. “The Health Commission wants thirty consecutive days of real-world data. If we skip two years, the timeline for the retrospective report breaks. The approval won’t pass.”

He took two bites, set down his chopsticks, and kept his eyes on the screen. In the exception queue, a batch of records kept triggering FIELD_MISSING. He opened a sample and found they were follow-up records for the same group of patients. The chief complaint field was empty, but the lab indicators were complete. The doctors back then had probably only ordered lab tests without writing clinical notes.

Lin Chen pulled up the ICD-10 mapping table and coded a rule: if the chief complaint was empty but the lab indicators matched a specific combination (e.g., fasting blood glucose ≥7.0 and HbA1c ≥6.5%), the system would auto-infer “suspected diabetes follow-up,” apply an INFERRED tag, and set the confidence level to 0.6. It wouldn’t replace a clinical diagnosis; it would just ensure the data chain remained unbroken.

He hit Enter. The pipeline swallowed the batch and moved forward.

At 9 PM, progress hit 64%. Lin Chen’s left foot began to twitch uncontrollably. He stopped typing, lowered his foot from the pad, and placed it flat on the floor. A sharp, crawling pain shot up his calf. He clenched his back teeth, making no sound. He pulled a bottle of ibuprofen from his drawer and dry-swallowed two pills.

Su Man hadn’t left. She was at the adjacent desk organizing supplementary materials for the ethics committee, occasionally glancing up at the screen.

“Your leg won’t hold out until Friday.” Her voice was light, devoid of persuasion, merely stating a fact.

“The pipeline is automated now,” Lin Chen said, watching the logs. “The rest is mostly manual review of the exception queue. I can just sit.”

“Sitting will cause necrosis too.” Su Man closed her folder. “Tomorrow I’ll go to the IT department and request the 2016 paper medical record archive catalog. Cross-referencing with the catalog will cut your inference workload in half.”

Lin Chen nodded. “Good. Take high-res photos of the catalog, sort them by timestamp, and send them to me.”

Su Man looked at him for two seconds, then turned and left. The server room door clicked shut.

Lin Chen placed his hands back on the keyboard. The cold glow of the monitor washed over his face. He opened the exception queue and began verifying records one by one. Some garbled text truly couldn’t be recovered; he could only preserve the raw snapshots and note in the report: “Historical system encoding lost; quarantined.” For missing fields, he reverse-engineered them from lab results and medication records. The work was dull, repetitive, and offered no shortcuts. But he was used to it. Ever since taking outsourcing gigs at county internet cafes, he had known that technology wasn’t magic. It was manual labor. It was taking a pile of mess and lining it up, line by line.

At 1 AM, progress broke 85%. The exception queue had dropped to 4,000.

Lin Chen rubbed his brow, preparing for a final batch validation. The moment he keyed in the command, the log window flashed a line of yellow text: WARNING: Timestamp collision detected. Batch 2019-Q3, 14,200 records share identical ingestion time.

His fingers froze in mid-air.

Fourteen thousand two hundred records, sharing the exact same ingestion time. Down to the millisecond.

In a real outpatient system, this was impossible. Unless it was a batch import, or the system clock had been manually reset. Lin Chen pulled up the raw files for this batch and found they all originated from a “historical data migration backup” at Provincial Second Hospital in September 2019. The migration log showed that, in a rush to launch the new system, the IT department’s outsourced team had used a script to perform a full overwrite.

What did an overwrite mean? It meant original visit serial numbers might have been rewritten, timelines compressed, and the anonymized data’s retrospective trail could be flagged by the Health Commission’s audit as “questionable data authenticity.”

Lin Chen stared at the yellow warning, his breathing steady. He didn’t panic. He opened his notebook and wrote on a fresh page: Batch overwrite. Timestamp collision. Audit risk.

He closed the warning window and created a new validation script. No fixes. No cover-ups. He isolated those 14,200 records, tagged them MIGRATION_ARTIFACT, and prepared to attach raw screenshots of the migration logs and cross-validation pathways to the final report.

Compliance wasn’t about perfection. It was about transparency.

He hit Enter. The script began to run. Outside the window, the city had fallen completely silent. Only the server fans kept spinning. Lin Chen leaned back in his chair and closed his eyes. His left foot still throbbed, but he didn’t move.

Tomorrow morning, Su Man would bring the paper catalog. In the afternoon, he would finish the final mapping round. The day after, the draft report.

Seven days. Five left.

The progress bar on the screen crawled forward. Lin Chen opened his eyes, his fingers returning to the keyboard. The next step was to verify the raw hash values of the migration logs.