Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hmtx/vmtx] >64k expansion #7

Closed
behdad opened this issue Sep 14, 2021 · 5 comments
Closed

[hmtx/vmtx] >64k expansion #7

behdad opened this issue Sep 14, 2021 · 5 comments
Labels
Implemented Implemented in HarfBuzz or doesn't need to >64k expansion

Comments

@behdad
Copy link
Member

behdad commented Sep 14, 2021

I talk about hmtx only. vmtx case is similar.

Currently hmtx table length is determined by hhea/maxp in the following way:

  • hhea.metricDataFormat is required to be 0 for current format;
  • hmtx is required to be 2 * hhea.numberOfHMetrics + 2 * maxp.numGlyphs bytes long.

The rest of this issue describes how we relax the second requirement, to allow encoding advance width of arbitrary number of glyphs in hmtx table, regardless of the value of maxp.numGlyphs.

This is how an advance width is assigned to every glyph index beyond maxp.numGlyphs:

  • Let B be the excess bytes at the end of the hmtx table beyond 2 * hhea.numberOfHMetrics + 2 * maxp.numGlyphs bytes.
  • If length of B is odd, ignore the last byte of B.
  • Treat B as an array of big-endian uint16 advance-width numbers of glyph indices starting at maxp.numGlyphs. For any glyph index that is in range in the font but out of range in this array, use the last item of the array.
@behdad behdad added this to the BE 1.0 ~= OT 2.0 milestone Sep 15, 2021
behdad added a commit to harfbuzz/harfbuzz that referenced this issue Feb 15, 2022
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Mar 8, 2022
Overview of changes leading to 4.0.0
Tuesday, March 1, 2022
====================================
- New public API to create subset plan and gather information on things like
  glyph mappings in the final subset. The plan can then be passed on to perform
  the subsetting operation. (Garret Rieger)
- Draw API for extracting glyph shapes have been extended and finalized and is
  no longer an experimental API. The draw API supports glyf, CFF and CFF2
  glyph outlines tables, and applies variation settings set on the font as well
  as synthetic slant. The new public API is not backward compatible with the
  previous, non-public, experimental API. (Behdad Esfahbod)
- The hb-view tool will use HarfBuzz draw API to render the glyphs instead of
  cairo-ft when compiled with Cairo 1.17.5 or newer, setting HB_DRAW
  environment variable to 1 or 0 will force using or not use the draw API,
  respectively. (Behdad Esfahbod)
- The hb-shape and hb-view tools now default to using HarfBuzz’s own font
  loading functions (ot) instead of FreeType ones (ft). They also have a new
  option, --font-slant, to apply synthetic slant to the font. (Behdad Esfahbod)
- HarfBuzz now supports more than 65535 (the OpenType limit) glyph shapes and
  metrics. See harfbuzz/boring-expansion-spec#6 and
  harfbuzz/boring-expansion-spec#7 for details.
  (Behdad Esfahbod)
- New API to get the dominant horizontal baseline tag for a given script.
  (Behdad Esfahbod)
- New API to get the baseline positions from the font, and synthesize missing
  ones. As well as new API to get font metrics and synthesize missing ones.
  (Matthias Clasen)
- Improvements to finding dependencies on Windows when building with Visual
  Studio. (Chun-wei Fan)
- New buffer flag, HB_BUFFER_FLAG_PRODUCE_UNSAFE_TO_CONCAT, that must be set
  during shaping for HB_GLYPH_FLAG_UNSAFE_TO_CONCAT flag to be reliably
  produced. This is to limit the performance hit of producing this flag to when
  it is actually needed. (Behdad Esfahbod)
- Documentation improvements. (Matthias Clasen)
mtremer pushed a commit to ipfire/ipfire-2.x that referenced this issue Apr 24, 2022
- Update from 3.4.0 to 4.2.0
- Update of rootfile
- Changelog
   Overview of changes leading to 4.2.0
     Wednesday, March 30, 2022
	- Source code reorganization, splitting large hb-ot-layout files into smaller,
	  per-subtable ones under OT/Layout/*. Code for more tables will follow suit in
	  later releases. (Garret Rieger, Behdad Esfahbod)
	- Revert Indic shaper change in previous release that broke some fonts and
	  instead make per-syllable restriction of “GSUB” application limited to
	  script-specific Indic features, while applying them and discretionary
	  features in one go. (Behdad Esfahbod)
	- Fix decoding of private in gvar table. (Behdad Esfahbod)
	- Fix handling of contextual lookups that delete too many glyphs. (Behdad Esfahbod)
	- Make “morx” deleted glyphs don’t block “GPOS” application. (Behdad Esfahbod)
	- Various build fixes. (Chun-wei Fan, Khaled Hosny)
	- New API
	   +hb_set_next_many() (Andrew John)
   Overview of changes leading to 4.1.0
     Wednesday, March 23, 2022
	- Various OSS-Fuzz fixes. (Behdad Esfahbod)
	- Make fallback vertical-origin match FreeType’s. (Behdad Esfahbod)
	- Treat visible viramas like dependent vowels in USE shaper. (David Corbett)
	- Apply presentation forms features and discretionary features in one go in
	  Indic shaper, which seems to match Uniscribe and CoreText behaviour.
	  (Behdad Esfahbod, David Corbett)
	- Various bug fixes.
	- New API
	   +hb_set_add_sorted_array() (Andrew John)
   Overview of changes leading to 4.0.1
     Friday, March 11, 2022
	- Update OpenType to AAT mappings for “hist” and “vrtr” features.
	  (Florian Pircher)
	- Update IANA Language Subtag Registry to 2022-03-02. (David Corbett)
	- Update USE shaper to allow any non-numeric tail in a symbol cluster, and
	  remove obsolete data overrides. (David Corbett)
	- Fix handling of baseline variations to return correctly scaled values.
	  (Matthias Clasen)
	- A new experimental hb_subset_repack_or_fail() to repack an array of objects,
	  eliminating offset overflows. The API is not available unless HarfBuzz is
	  built with experimental APIs enabled. (Qunxin Liu)
	- New experimental API
	   +hb_link_t
	   +hb_object_t
	   +hb_subset_repack_or_fail()
   Overview of changes leading to 4.0.0
     Tuesday, March 1, 2022
	- New public API to create subset plan and gather information on things like
	  glyph mappings in the final subset. The plan can then be passed on to perform
	  the subsetting operation. (Garret Rieger)
	- Draw API for extracting glyph shapes have been extended and finalized and is
	  no longer an experimental API. The draw API supports glyf, CFF and CFF2
	  glyph outlines tables, and applies variation settings set on the font as well
	  as synthetic slant. The new public API is not backward compatible with the
	  previous, non-public, experimental API. (Behdad Esfahbod)
	- The hb-view tool will use HarfBuzz draw API to render the glyphs instead of
	  cairo-ft when compiled with Cairo 1.17.5 or newer, setting HB_DRAW
	  environment variable to 1 or 0 will force using or not use the draw API,
	  respectively. (Behdad Esfahbod)
	- The hb-shape and hb-view tools now default to using HarfBuzz’s own font
	  loading functions (ot) instead of FreeType ones (ft). They also have a new
	  option, --font-slant, to apply synthetic slant to the font. (Behdad Esfahbod)
	- HarfBuzz now supports more than 65535 (the OpenType limit) glyph shapes and
	  metrics. See harfbuzz/boring-expansion-spec#6 and
	  harfbuzz/boring-expansion-spec#7 for details.
	  (Behdad Esfahbod)
	- New API to get the dominant horizontal baseline tag for a given script.
	  (Behdad Esfahbod)
	- New API to get the baseline positions from the font, and synthesize missing
	  ones. As well as new API to get font metrics and synthesize missing ones.
	  (Matthias Clasen)
	- Improvements to finding dependencies on Windows when building with Visual
	  Studio. (Chun-wei Fan)
	- New buffer flag, HB_BUFFER_FLAG_PRODUCE_UNSAFE_TO_CONCAT, that must be set
	  during shaping for HB_GLYPH_FLAG_UNSAFE_TO_CONCAT flag to be reliably
	  produced. This is to limit the performance hit of producing this flag to when
	  it is actually needed. (Behdad Esfahbod)
	- Documentation improvements. (Matthias Clasen)
	- New API
	 - General:
	   +HB_BUFFER_FLAG_PRODUCE_UNSAFE_TO_CONCAT
	   +hb_var_num_t
	 - Draw:
	   +hb_draw_funcs_t
	   +hb_draw_funcs_create()
	   +hb_draw_funcs_reference()
	   +hb_draw_funcs_destroy()
	   +hb_draw_funcs_is_immutable()
	   +hb_draw_funcs_make_immutable()
	   +hb_draw_move_to_func_t
	   +hb_draw_funcs_set_move_to_func()
	   +hb_draw_line_to_func_t
	   +hb_draw_funcs_set_line_to_func()
	   +hb_draw_quadratic_to_func_t
	   +hb_draw_funcs_set_quadratic_to_func()
	   +hb_draw_cubic_to_func_t
	   +hb_draw_funcs_set_cubic_to_func()
	   +hb_draw_close_path_func_t
	   +hb_draw_funcs_set_close_path_func()
	   +hb_draw_state_t
	   +HB_DRAW_STATE_DEFAULT
	   +hb_draw_move_to()
	   +hb_draw_line_to()
	   +hb_draw_quadratic_to()
	   +hb_draw_cubic_to()
	   +hb_draw_close_path()
	   +hb_font_get_glyph_shape_func_t
	   +hb_font_funcs_set_glyph_shape_func()
	   +hb_font_get_glyph_shape()
	 - OpenType layout
	   +HB_OT_LAYOUT_BASELINE_TAG_IDEO_FACE_CENTRAL
	   +HB_OT_LAYOUT_BASELINE_TAG_IDEO_EMBOX_CENTRAL
	   +hb_ot_layout_get_horizontal_baseline_tag_for_script()
	   +hb_ot_layout_get_baseline_with_fallback()
	 - Metrics:
	   +hb_ot_metrics_get_position_with_fallback()
	 - Subset:
	   +hb_subset_plan_t
	   +hb_subset_plan_create_or_fail()
	   +hb_subset_plan_reference()
	   +hb_subset_plan_destroy()
	   +hb_subset_plan_set_user_data()
	   +hb_subset_plan_get_user_data()
	   +hb_subset_plan_execute_or_fail()
	   +hb_subset_plan_unicode_to_old_glyph_mapping()
	   +hb_subset_plan_new_to_old_glyph_mapping()
	   +hb_subset_plan_old_to_new_glyph_mapping()

Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
behdad added a commit to fonttools/fonttools that referenced this issue Jul 3, 2022
behdad added a commit to fonttools/fonttools that referenced this issue Jul 8, 2022
behdad added a commit to fonttools/fonttools that referenced this issue Jul 12, 2022
behdad added a commit to fonttools/fonttools that referenced this issue Jul 12, 2022
@behdad behdad added the Implemented Implemented in HarfBuzz or doesn't need to label Jul 12, 2022
behdad added a commit to fonttools/fonttools that referenced this issue Jul 12, 2022
behdad added a commit to fonttools/fonttools that referenced this issue Jul 12, 2022
@behdad
Copy link
Member Author

behdad commented Mar 6, 2023

@behdad behdad closed this as completed Mar 6, 2023
@lrosenthol
Copy link

But this algorithm is completely different from that used by every other font processor in existence today - which means it will break compatibility with all those other tools....and that doesn't seem like a reasonable approach.

@behdad
Copy link
Member Author

behdad commented Apr 14, 2023

But this algorithm is completely different from that used by every other font processor in existence today - which means it will break compatibility with all those other tools....and that doesn't seem like a reasonable approach.

It's not completely different. If gid is less than maxp.numGlyphs the two algorithms produce the same output. That's backward compatibility right there. Indeed, HarfBuzz uses this algorithm when compiled with experimental-apis feature.

Can you elaborate what problem you see with this?

@lrosenthol
Copy link

You are assuming that all the processor does is to index into the array - it doesn't copy or evaluate the bytes of the htmx otherwise, and I don't believe that to be true.

Certainly, any software doing data validation will fail because the htmx block will be larger than expected.

Why not simply leave the htmx (and vtmx) alone and create new extended tables for the values beyond the current max?

@behdad
Copy link
Member Author

behdad commented Apr 14, 2023

I think an older font sanitizer failing against a newer font format is naturally expected.

Why not simply leave the htmx (and vtmx) alone and create new extended tables for the values beyond the current max?

That's possible, but I don't see the problem with what I'm proposing. You can make the same argument about extending any table. With your argument we cannot extend anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Implemented Implemented in HarfBuzz or doesn't need to >64k expansion
Projects
None yet
Development

No branches or pull requests

2 participants