Hugging Face reposted this
Are current open OCR models good enough to unlock historical knowledge? Daniel van Strien and I have been working on this question for the last few months — and today we are proud to introduce FineBooks, a new collaboration between Hugging Face and EleutherAI. Our first release is the BHL OCR Leaderboard: 14 open OCR models scored against 2,165 historical book pages transcribed by expert researchers from six volumes of the Biodiversity Heritage Library — in English, French, German and Latin. https://lnkd.in/eguACYSR The ground truth behind the leaderboard was produced in 2011–12 by the IMPACT project and BHL-Europe, released under CC-BY, and preserved by IMPACT's technical manager Clemens Neudecker on Github after both projects ended. This is what open science is all about: releasing your data openly so that down the line someone can re-use it for purposes you never planned for. Exactly the same open science and open data philosophy is at the core of FineBooks. The next step: we'll re-run OCR across the roughly 200,000 public domain volumes from the BHL and release the improved text openly as the first FineBooks dataset.