flow_preprocessing.preprocessing_logic.preprocess.PreprocessorBuilder

class flow_preprocessing.preprocessing_logic.preprocess.PreprocessorBuilder(huggingface_target_repo_name)

Builder for creating Preprocessor instances with fluent API.

Makes it easier to create preprocessors with complex configurations without needing to manually create PreprocessorConfig objects.

Example:

preprocessor = (PreprocessorBuilder(“username/output-dataset”)

.with_token(“hf_token”) .with_export_mode(“line”) .with_segmentation(segmenter_config) .with_split(0.8) .build_for_zip(“data.zip”))

Parameters:

huggingface_target_repo_name (str)

__init__(huggingface_target_repo_name)

Initialize builder.

Parameters:

huggingface_target_repo_name (str) – Target HuggingFace repository.

Methods

__init__(huggingface_target_repo_name)	Initialize builder.
append([should_append])	Set append mode.
build_for_huggingface(repo_id)	Build a HuggingFacePreprocessor.
build_for_zip(zip_path)	Build a ZipPreprocessor.
private([is_private])	Set repository privacy.
with_batch_size(batch_size)	Set batch size for processing.
with_crop([crop])	Enable image cropping.
with_export_mode(mode)	Set export mode.
with_line_filtering([min_width, min_height])	Configure line filtering.
with_segmentation(segmenter_config[, backend])	Enable image segmentation.
with_split(ratio[, seed, shuffle])	Configure dataset splitting.
with_token(token)	Set HuggingFace token.