The first challenge was matching the crop borders, easier said than done because "dense Urdu print repeats strokes, words, borders, and column patterns," meaning that marks from neighboring pages could throw off the measurements. Next up, visible area identification and gutter separation were made tricky by tables and deep gutter shadows. Cropping errors required a specific pass, as did blemish removal.
Interestingly enough, the researcher noticed that using a bigger model or more training samples actually reduced the accuracy of the results. The choice of crop margin (inset) for each book was unique, made to the editor's judgement. But since it had no discernible pattern, adding more books made the model's pattern recognition worse, not better.
The final process still needs ten calibration crops for a new book, but that's a pretty reasonable amount of manual labor to be able to process a book entirely. Finally, the books are hosted in a ZFS pool with BLAKE3 manifests.
The entire tale and all the technical details are explained in a lengthy blog post , and you can peruse the beautiful Urdu writings at the Ibteda Digital Library page at the Internet Archive . My Western eyes can't read any of it, but the poem translations are beautiful.
Follow Tom's Hardware on Google News , or add us as a preferred source , to get our latest news, analysis, & reviews in your feeds.
Bruno Ferreira is a contributing writer for Tom's Hardware. He has decades of experience with PC hardware and assorted sundries, alongside a career as a developer. He's obsessed with detail and has a tendency to ramble on the topics he loves. When not doing that, he's usually playing games, or at live music shows and festivals. ","collapsible":{"enabled":true,"maxHeight":250,"readMoreText":"Read more","readLessText":"Read less"}}), "https://slice.vanilla.futurecdn.net/13-4-25/js/authorBio.js"); } else { console.error('%c FTE ','background: #9306F9; color: #ffffff','no lazy slice hydration function available'); } Bruno Ferreira Social Links Navigation Contributor Bruno Ferreira is a contributing writer for Tom's Hardware. He has decades of experience with PC hardware and assorted sundries, alongside a career as a developer. He's obsessed with detail and has a tendency to ramble on the topics he loves. When not doing that, he's usually playing games, or at live music shows and festivals.
Hooda Thunkett Why did they have to quit? I'd love for them to continue, or for someone else to pick up the baton! Reply
jackt This is horrible, manage old rare books to make low quality "scans" to feed AI is horrible. Reply
usertests jackt said: This is horrible, manage old rare books to make low quality "scans" to feed AI is horrible. Yeah, we should just burn all of these books, so AI can't get them. Reply
Misgar jackt said: This is horrible, manage old rare books to make low quality "scans" to feed AI is horrible. The goal wasn't "to make low quality "scans" to feed AI". Instead they scanned rare books and used AI to make them available to anyone with internet access. There's a big difference between scanning to train AI (and destroying the books in the process) versus using AI to make rare texts freely available. Despite squashing the books under a glass plate to flatten the pages, I doubt they completely destroyed the books in the process. A V-scanner would be better, but they were on a shoehorn budget. As for "low quality", the D3300 and D5300 have 24MP sensors capable of 6,000 by 4,000 pixels. By my calculation, a page measuring 12" by 8" can be scanned at 500 DPI. If you consider Urdu script to be "Artwork" 500dpi is below the 600DPI Minimum, but for "Text", 500dpi is in the middle of the 400-600DPI Recommended range. https://techlasi.com/savvy/document-scanning-best-practices-for-archivists/ Optimal Scanning Settings and Specifications Technical specifications directly impact digital file usability and preservation value. Proper settings balance file size constraints with image quality requirements for long-term accessibility. Document Type Minimum DPI Recommended DPI Color Mode Text documents 300 400-600 Grayscale/Color Photographs 600 1200 Color Maps/Technical drawings 400 800-1200 Color Microfilm 600 1000+ Grayscale Artwork 600 1200-2400 Color Reply
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/artificial-intelligence/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/artificial-intelligence/diy-archivists-push-budget-nikons-to-902-000-clicks-to-save-1-800-rare-books-team-trains-neural-net-on-photoshop-edits-to-process-526-000-scans#main
- https://www.tomshardware.com/membership
- NVIDIA NVLink Fusion Expands With NVHBM Custom High-Bandwidth Memory
- Massive 12TB Steam leak reveals decades of unreleased games — archived files include unseen Half-Life 2: Episode 3 builds and assets
- Securing the Infrastructure of Intelligence
- Speedy 34-inch 240 Hz ultrawide OLED monitor now $600 off — LG UltraGear 34GX900A-B only $599.99
- NVIDIA NVLink Fusion Expands With NVHBM Custom High-Bandwidth Memory
Informational only. No financial advice. Do your own research.