sc_crawler.logger

Classes:

• ScRichHandler – Extend RichHandler with function name logged in the right column.
• VendorProgressTracker – Tracking the progress of the vendor's inventory updates.
• VoidProgressTracker – Progress tracker reference not doing antyhing.

Functions:

• log_start_end – Log the start and end of the decorated function.

func log_start_end

log_start_end(func)

Log the start and end of the decorated function.

class ScRichHandler

Bases: RichHandler

Extend RichHandler with function name logged in the right column.

class VendorProgressTracker

Tracking the progress of the vendor's inventory updates.

Functions:

• start_vendor – Starts a progress bar for the Vendor's steps.
• advance_vendor – Increment the number of finished steps.
• update_vendor – Update the vendor's progress bar.
• start_task – Starts a progress bar in the list of current jobs.
• last_task – Returh the last registered TaskID.
• advance_task – Increment the number of finished steps.
• update_task – Update the task's progress bar.
• hide_task – Hide a task from the list of progress bars.

Attributes:

• task_ids (List[TaskID]) – List of active task ids for the current vendor.
• vendor (Vendor) – A sc_crawler.tables.Vendor instance for which tracking progress.
• progress_panel (ProgressPanel) – A rich panel including progress bars.
• vendors (Progress) – rich.progress.Progress for tracking the inventory steps of the vendor.
• tasks (Progress) – rich.progress.Progress for tracking the lower-level tasks within each step.
• metadata (Text) – rich.text.Text metadata, e.g. data sources and records to be udpated.

attr

VendorProgressTracker.task_ids

task_ids: List[TaskID] = []

List of active task ids for the current vendor.

attr

VendorProgressTracker.vendor

vendor: Vendor = vendor

A sc_crawler.tables.Vendor instance for which tracking progress.

attr

VendorProgressTracker.progress_panel

progress_panel: ProgressPanel = progress_panel

A rich panel including progress bars. Should not be used directly, see the vendors, tasks and metadata attributes.

attr

VendorProgressTracker.vendors

vendors: Progress = progress_panel.vendors

rich.progress.Progress for tracking the inventory steps of the vendor.

attr

VendorProgressTracker.tasks

tasks: Progress = progress_panel.tasks

rich.progress.Progress for tracking the lower-level tasks within each step.

attr

VendorProgressTracker.metadata

metadata: Text = progress_panel.metadata

rich.text.Text metadata, e.g. data sources and records to be udpated.

meth

VendorProgressTracker.start_vendor

start_vendor(total)

Starts a progress bar for the Vendor's steps.

Parameters:

• total (int) – Overall number of steps to show in the progress bar.

Returns:

• TaskId (TaskID) – The progress bar's identifier to be referenced in future updates.

meth

VendorProgressTracker.advance_vendor

advance_vendor(advance=1)

Increment the number of finished steps.

Parameters:

• advance (int) – Number of steps to advance.

meth

VendorProgressTracker.update_vendor

update_vendor(**kwargs)

Update the vendor's progress bar.

Useful fields:

• step: Name of the currently running step to be shown on the progress bar.

meth

VendorProgressTracker.start_task

start_task(name, total)

Starts a progress bar in the list of current jobs.

Besides returning the TaskID, it will also register in self.tasks.task_ids as the last task, which will be the default value for future advance_task, hide_task etc calls. The latter will remove the TaskID from the task_ids.

Parameters:

• name (str) – Name to show in front of the progress bar. Will be prefixed by Vendor's name.
• total (int) – Overall number of steps to show in the progress bar.

Returns:

• TaskId (TaskID) – The progress bar's identifier to be referenced in future updates.

meth

VendorProgressTracker.last_task

last_task()

Returh the last registered TaskID.

meth

VendorProgressTracker.advance_task

advance_task(task_id=None, advance=1)

Increment the number of finished steps.

Parameters:

• task_id (Optional[TaskID]) – The progress bar's identifier returned by start_task. Defaults to the most recently created task.
• advance (int) – Number of steps to advance.

meth

VendorProgressTracker.update_task

update_task(task_id=None, **kwargs)

Update the task's progress bar.

Parameters:

• task_id (Optional[TaskID]) – The progress bar's identifier returned by start_task. Defaults to the most recently created task.

Other Parameters:

• step (str) – Name of the currently running step to be shown on the progress bar.

See rich.progress.Progress.update for further keyword arguments.

meth

VendorProgressTracker.hide_task

hide_task(task_id=None)

Hide a task from the list of progress bars.

Parameters:

• task_id (Optional[TaskID]) – The progress bar's identifier returned by start_task. Defaults to the most recently created task.

class VoidProgressTracker

Bases: VendorProgressTracker

Progress tracker reference not doing antyhing.

Functions:

• start_vendor – Starts a progress bar for the Vendor's steps.
• advance_vendor – Increment the number of finished steps.
• update_vendor – Update the vendor's progress bar.
• start_task – Starts a progress bar in the list of current jobs.
• last_task – Returh the last registered TaskID.
• advance_task – Increment the number of finished steps.
• update_task – Update the task's progress bar.
• hide_task – Hide a task from the list of progress bars.

meth

VoidProgressTracker.start_vendor

start_vendor(*args, **kwargs)

Starts a progress bar for the Vendor's steps.

Parameters:

• total (int) – Overall number of steps to show in the progress bar.

Returns:

• TaskId (TaskID) – The progress bar's identifier to be referenced in future updates.

meth

VoidProgressTracker.advance_vendor

advance_vendor(*args, **kwargs)

Increment the number of finished steps.

Parameters:

• advance (int) – Number of steps to advance.

meth

VoidProgressTracker.update_vendor

update_vendor(*args, **kwargs)

Update the vendor's progress bar.

Useful fields:

• step: Name of the currently running step to be shown on the progress bar.

meth

VoidProgressTracker.start_task

start_task(*args, **kwargs)

Starts a progress bar in the list of current jobs.

Besides returning the TaskID, it will also register in self.tasks.task_ids as the last task, which will be the default value for future advance_task, hide_task etc calls. The latter will remove the TaskID from the task_ids.

Parameters:

• name (str) – Name to show in front of the progress bar. Will be prefixed by Vendor's name.
• total (int) – Overall number of steps to show in the progress bar.

Returns:

• TaskId (TaskID) – The progress bar's identifier to be referenced in future updates.

meth

VoidProgressTracker.last_task

last_task(*args, **kwargs)

Returh the last registered TaskID.

meth

VoidProgressTracker.advance_task

advance_task(*args, **kwargs)

Increment the number of finished steps.

Parameters:

• task_id (Optional[TaskID]) – The progress bar's identifier returned by start_task. Defaults to the most recently created task.
• advance (int) – Number of steps to advance.

meth

VoidProgressTracker.update_task

update_task(*args, **kwargs)

Update the task's progress bar.

Parameters:

• task_id (Optional[TaskID]) – The progress bar's identifier returned by start_task. Defaults to the most recently created task.

Other Parameters:

• step (str) – Name of the currently running step to be shown on the progress bar.

See rich.progress.Progress.update for further keyword arguments.

meth

VoidProgressTracker.hide_task

hide_task(*args, **kwargs)

Hide a task from the list of progress bars.

Parameters:

• task_id (Optional[TaskID]) – The progress bar's identifier returned by start_task. Defaults to the most recently created task.