NeoAnd/Docling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fit: Specify encoding when writing output file (#214)

Browse Source 
Specify encoding when writing output file to avoid errors when default target encoding doesn't have all characters. utf8 seems like the most universal and supported encoding. Otherwise, the cli fails with encoding errors when input file contains unicode text (basically most files nowadays) and the target system has default encoding set to some one-byte charset like cp1252

Signed-off-by: Johnny Salazar <cepera.ang@gmail.com>
This commit is contained in:
Johnny Salazar 2024-11-04 20:24:13 +07:00 committed by GitHub
parent 8fb445f46c
commit af323c04ef
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docling/cli/main.py
View File

@ -90,28 +90,28 @@ def export_documents(
# Export Deep Search document JSON format:
if export_json:
fname = output_dir / f"{doc_filename}.json"
with fname.open("w") as fp:
with fname.open("w", encoding="utf8") as fp:
_log.info(f"writing JSON output to {fname}")
fp.write(json.dumps(conv_res.document.export_to_dict()))
# Export Text format:
if export_txt:
fname = output_dir / f"{doc_filename}.txt"
with fname.open("w") as fp:
with fname.open("w", encoding="utf8") as fp:
_log.info(f"writing Text output to {fname}")
fp.write(conv_res.document.export_to_markdown(strict_text=True))
# Export Markdown format:
if export_md:
fname = output_dir / f"{doc_filename}.md"
with fname.open("w") as fp:
with fname.open("w", encoding="utf8") as fp:
_log.info(f"writing Markdown output to {fname}")
fp.write(conv_res.document.export_to_markdown())
# Export Document Tags format:
if export_doctags:
fname = output_dir / f"{doc_filename}.doctags"
with fname.open("w") as fp:
with fname.open("w", encoding="utf8") as fp:
_log.info(f"writing Doc Tags output to {fname}")
fp.write(conv_res.document.export_to_document_tokens())