fix: Test cases for RTL programmatic PDFs and fixes for the formula model (#903)

fix: Support for RTL programmatic documents
fix(parser): detect and handle rotated pages
fix(parser): fix bug causing duplicated text
fix(formula): improve stopping criteria
chore: update lock file
fix: temporary constrain beautifulsoup


* switch to code formula model v1.0.1 and new test pdf

Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>

* switch to code formula model v1.0.1 and new test pdf

Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>

* cleaned up the data folder in the tests

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* switch to code formula model v1.0.1 and new test pdf

Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>

* added three test-files for right-to-left

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* fix black

Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>

* added new gt for test_e2e_conversion

Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>

* added new gt for test_e2e_conversion

Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>

* Add code to expose text direction of cell

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* new test file

Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>

* update lock

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* fix mypy reports

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* fix example filepaths

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* add test data results

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* pin wheel of latest docling-parse release

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* use latest docling-core

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* remove debugging code

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* fix path to files in example

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* Revert unwanted RTL additions

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fix test data paths in examples

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Matteo-Omenetti <Matteo.Omenetti1@ibm.com>
Co-authored-by: Peter Staar <taa@zurich.ibm.com>
Co-authored-by: Christoph Auer <cau@zurich.ibm.com>

tests/data/groundtruth/docling_v1/right_to_left_03.md

@@ -0,0 +1,55 @@
+## یلخاد یلااک - یلصا رازاب رد شريذپ همانديما
+
+<!-- image -->
+
+## لااک درادناتسا -2-5
+
+درادناتسا مان
+
+یرگ هتخير شور هب هدش ديلوت لاشمش و هشمش فرصم دروم هتسويپ یا هزاس یاهدلاوف رد - قباطم تسويپ زيلانآ
+
+یلم درادناتسا هرامش
+
+20300
+
+؟تسا یرابجا درادناتسا
+
+درادناتسا هدننکرداص عجرم
+
+ناريا درادناتسا یلم نامزاس
+
+ذخا ار روکذم درادناتسا ،لوصحم هدننکديلوت ايآ ؟تسا هدومن
+
+## سروب رد شريذپ -3
+
+کرادم هئارا خيرات
+
+1403/09/19
+
+شريذپ خيرات
+
+1403/10/04
+
+هضرع هتيمک هسلج هرامش
+
+436
+
+همانديما جرد خيرات
+
+1403/10/05
+
+شريذپ رواشم
+
+سروب نومرآ یرازگراک
+
+رد لااک شريذپ زا سپ هياپ تميق نييعت ةوحن سروب
+
+یناهج  یاه تميق ساسا رب
+
+شورف /شورف لک /ديلوت زا هضرع دصرد لقادح یلخاد
+
+نت 47.500 اي هنايلاس ديلوت زا %50 لقادح
+
+ليوحت زاجم یاطخ
+
+ليوحت لباق هلومحم نيرخآ 5%