How to convert handwritten notes to text

Handwriting is the hardest everyday job for OCR. Neat, printed notes can come out well with a few corrections. Joined-up cursive often comes out as a mix of right words and nonsense. Knowing which kind of notes you have, and how to capture them, decides whether OCR saves you time or costs you more.

This guide sets honest expectations, then covers capture, review, and the cases where typing your notes up is simply quicker.

What OCR can and can’t read

Most OCR engines, including the open-source Tesseract engine, learned to read from printed text in a wide range of fonts. Handwriting varies far more: from person to person, from page to page, and even between two copies of the same letter in one sentence. How close your writing is to print matters more than anything else.

Print-style handwriting

Separate letters with clear gaps between words are the best case. Block capitals are often the easiest of all. Expect most words to come through, with errors clustered around letters that look alike in your hand: a and o, u and n, r and v, 1 and 7, 4 and 9.

Joined-up cursive

In cursive, letters run into each other, so the engine can’t see where one ends and the next begins. Loops and connecting strokes look like extra letters. A general-purpose OCR engine will usually get some words right and garble the rest.

Systems built specifically for handwriting, known as handwritten text recognition (HTR), are trained on handwriting samples and cope better with cursive, though they make mistakes too. If most of your notes are cursive and you need them as text, set aside time for correction or consider the alternatives further down.

Everything that isn’t a line of text

Arrows, mind maps, boxes, circled words, crossed-out phrases and notes squeezed into the margin don’t fit the line-by-line way OCR reads a page. They’re usually skipped or turned into stray characters. Diagrams and sketches need to stay as images.

Capturing handwritten pages

The general rules for good photos apply here, and they’re covered in how to get accurate OCR results. Handwriting adds a few problems of its own.

• Pencil is faint and shiny. Graphite reflects light, so pencil notes can wash out in a photo. Take them in soft, even light, and raise the contrast afterward.
• Ruled lines get in the way. Lines that cut through descenders (the tails of g, y and p) can merge with the letters. Converting to grayscale and raising the contrast often fades light blue rules more than dark ink.
• Show-through confuses the engine. On thin paper written on both sides, the back page shows through. If you’re scanning, put a sheet of black paper behind the page to hide it.
• Notebooks curve near the spine. Lines bend as they approach the binding. Press the notebook flat and photograph one page at a time rather than a two-page spread.
• Whiteboards glare. Shoot from slightly to one side to move the reflection off the writing, then straighten the photo with a four-corner perspective tool. Red and green markers are often fainter than black or blue.
• Sticky notes need contrast. Lay them on a darker surface so their edges are clear, and photograph them close enough that the writing fills the frame.

Reading and reviewing the result

Review is where handwriting OCR succeeds or fails, so make it efficient.

1. Use a handwriting setting if your tool has one. In Image to Text App, the handwriting to text page uses a Handwriting mode with preprocessing tuned for notes. It works best on neat, print-style writing.
2. Expect a lower confidence level. A “fair” or “low” overall rating is normal for handwriting. It tells you to read carefully, not that the result is useless.
3. Work through the flagged words. Words the engine was unsure about are underlined. Jump from one to the next and fix each against the image.
4. Use the image when a word won’t make sense. Clicking a line of text highlights its spot on the photo, so you can look at the original handwriting without hunting for it.
5. Fix repeated mistakes in one go. If your handwritten “a” keeps coming out as “o”, find and replace can save time. Check each replacement, because a blanket change will also hit words that were read correctly.
6. Mark what you can’t read. If you can’t decipher a word even in the image, type a marker such as [?] rather than guessing. A wrong guess looks right later.

Review while the notes are fresh. You’ll fill in half-legible words from memory far more reliably today than in a month.

Typing up versus OCR

OCR isn’t always the fastest route from paper to text. A rough guide:

• A few lines. Just type them. You’ll finish before you’ve taken the photo.
• Pages of neat print. OCR and review is usually faster than typing, especially if you’re not a quick typist.
• Pages of messy cursive. Correction can take longer than retyping. Try dictation instead: read your notes aloud into the voice typing built into Windows (Windows+H), macOS, iOS or Android. You’re the best reader of your own handwriting, and for many people speaking is quicker than typing.
• Notes you mainly want to search later. Even imperfect OCR helps here. Save the page as a searchable PDF, which keeps the image and adds an invisible text layer, and you can find most notes by keyword while still seeing the original handwriting.
• Notes you’ll edit and share. OCR the readable parts, fix them, and export a Word file with the image to Word tool, then tidy the formatting there.

Writing notes that convert well

If you know your notes will end up as text, a few habits make a big difference:

• Print rather than join your letters, and leave clear space between words.
• Use a dark pen. A felt-tip or gel pen photographs better than a ballpoint or pencil.
• Write in a single column, and keep notes inside the margins.
• Write headings in capitals on their own line, and start list items with a dash.
• Number your pages so they’re easy to put back in order.
• Keep diagrams on a separate part of the page from the text.

None of this guarantees a perfect result, but it moves your notes much closer to the printed text that OCR engines read best. For a look at why some letters get confused and how confidence scores are worked out, see how OCR works.