flow_preprocessing.preprocessing_logic.config.PreprocessorBaseConfig

class flow_preprocessing.preprocessing_logic.config.PreprocessorBaseConfig(*, huggingface_target_repo_name, huggingface_target_repo_private=False, append=False, export_mode='line', crop=False, allow_empty_lines=False, batch_size=32, augmentation_loops=None, min_width_line=None, min_height_line=None, split_train_ratio=None, split_seed=42, split_shuffle=True, segment=None, segmenter_config=None)[source]

Configuration for the preprocessor.

Encapsulates all configuration parameters with validation and clear defaults. Uses Pydantic for validation and schema metadata.

Parameters:

  • huggingface_target_repo_name (Annotated[str, FieldInfo(annotation=NoneType, required=True, alias='huggingface_target_repo_name', alias_priority=2, title='HuggingFace-Target-Repo-Name', description='HuggingFace target repository name.', examples=['my-org/my-repo'])])

  • huggingface_target_repo_private (Annotated[bool | None, FieldInfo(annotation=NoneType, required=False, default=False, alias='huggingface_target_repo_private', alias_priority=2, title='HuggingFace-Target-Repo-Private', description='Whether the target repository is private.', examples=['false'])])

  • append (Annotated[bool | None, FieldInfo(annotation=NoneType, required=False, default=False, alias='append', alias_priority=2, title='Append', description='Append to an existing dataset if it exists, else overwrite it.', examples=['false'])])

  • export_mode (Annotated[str | None, FieldInfo(annotation=NoneType, required=False, default='line', alias='export_mode', alias_priority=2, title='Export-Mode', description='Export mode for preprocessing output.', examples=['line'])])

  • crop (Annotated[bool | None, FieldInfo(annotation=NoneType, required=False, default=False, alias='crop', alias_priority=2, title='Crop', description='Whether to crop lines during preprocessing.', examples=['false'])])

  • allow_empty_lines (Annotated[bool | None, FieldInfo(annotation=NoneType, required=False, default=False, alias='allow_empty_lines', alias_priority=2, title='Allow-Empty-Lines', description='Allow empty lines in output.', examples=['false'])])

  • batch_size (Annotated[int | None, FieldInfo(annotation=NoneType, required=False, default=32, alias='batch_size', alias_priority=2, title='Batch-Size', description='Batch size for processing.', examples=['32'])])

  • augmentation_loops (Annotated[int | None, FieldInfo(annotation=NoneType, required=False, default=None, alias='augmentation_loops', alias_priority=2, title='Augmentation-Loops', description="Augmentation loops for random preprocessing, works only with export_mode==line (e.g., if it is equal to 2, you'll get three lines)", examples=['2'], metadata=[Ge(ge=0)])])

  • min_width_line (Annotated[int | None, FieldInfo(annotation=NoneType, required=False, default=None, alias='min_width_line', alias_priority=2, title='Min-Width-Line', description='Minimum width of a line; None disables filtering.', examples=['40'], metadata=[Gt(gt=0)])])

  • min_height_line (Annotated[int | None, FieldInfo(annotation=NoneType, required=False, default=None, alias='min_height_line', alias_priority=2, title='Min-Height-Line', description='Minimum height of a line; None disables filtering.', examples=['10'], metadata=[Gt(gt=0)])])

  • split_train_ratio (Annotated[float | None, FieldInfo(annotation=NoneType, required=False, default=None, alias='split_train_ratio', alias_priority=2, title='Split-Train-Ratio', description='Train split ratio; None disables splitting.', examples=['0.8'], metadata=[Gt(gt=0.0), Le(le=1.0)])])

  • split_seed (Annotated[int, FieldInfo(annotation=NoneType, required=False, default=42, alias='split_seed', alias_priority=2, title='Split-Seed', description='Random seed for dataset splitting.', examples=['42'], metadata=[Ge(ge=0)])])

  • split_shuffle (Annotated[bool | None, FieldInfo(annotation=NoneType, required=False, default=True, alias='split_shuffle', alias_priority=2, title='Split-Shuffle', description='Shuffle before splitting the dataset.', examples=['true'])])

  • segment (Annotated[Literal['yolo', 'kraken'] | None, FieldInfo(annotation=NoneType, required=False, default=None, alias='segment', alias_priority=2, title='Segment', description='Segmentation backend to use.', examples=['yolo'])])

  • segmenter_config (Annotated[SegmenterConfig | SegmenterBaseConfig | dict | None, FieldInfo(annotation=NoneType, required=False, default=None, alias='segmenter_config', alias_priority=2, title='Segmenter-Config', description='Configuration for the segmentation backend.', examples=["{'model_names': 'Riksarkivet/yolov9-lines-within-regions-1','batch_sizes': 16,'order_lines': false,'baselines': true,'kraken_linemasks': true,'creator': 'yourname','load_existing_segmentation': true}"])])

__init__(**data)

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

data (Any)

Return type:

None

Methods

__init__(**data)

Create a new model by parsing and validating input data from keyword arguments.

construct([_fields_set])

copy(*[, include, exclude, update, deep])

Returns a copy of the model.

dict(*[, include, exclude, by_alias, ...])

from_orm(obj)

json(*[, include, exclude, by_alias, ...])

model_construct([_fields_set])

Creates a new instance of the Model class with validated data.

model_copy(*[, update, deep])

!!! abstract "Usage Documentation"

model_dump(*[, mode, include, exclude, ...])

!!! abstract "Usage Documentation"

model_dump_json(*[, indent, ensure_ascii, ...])

!!! abstract "Usage Documentation"

model_json_schema([by_alias, ref_template, ...])

Generates a JSON schema for a model class.

model_parametrized_name(params)

Compute the class name for parametrizations of generic classes.

model_post_init(context, /)

Override this method to perform additional initialization after __init__ and model_construct.

model_rebuild(*[, force, raise_errors, ...])

Try to rebuild the pydantic-core schema for the model.

model_validate(obj, *[, strict, extra, ...])

Validate a pydantic model instance.

model_validate_json(json_data, *[, strict, ...])

!!! abstract "Usage Documentation"

model_validate_strings(obj, *[, strict, ...])

Validate the given object with string data against the Pydantic model.

parse_file(path, *[, content_type, ...])

parse_obj(obj)

parse_raw(b, *[, content_type, encoding, ...])

schema([by_alias, ref_template])

schema_json(*[, by_alias, ref_template])

update_forward_refs(**localns)

validate(value)

Attributes

model_computed_fields

model_config

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_extra

Get extra fields set during validation.

model_fields

model_fields_set

Returns the set of fields that have been explicitly set on this model instance.

requires_xml_parsing

Check if the export mode requires XML parsing.

huggingface_target_repo_name

huggingface_target_repo_private

append

export_mode

crop

allow_empty_lines

batch_size

augmentation_loops

min_width_line

min_height_line

split_train_ratio

split_seed

split_shuffle

segment

segmenter_config