Split up parse_repo and apply_total

Is your feature request related to a problem? Please describe. I have a framework that handles the offline store. It creates the tables, indexes, reads data from different data sources, does some transformations, and then inserts into the offline store. As a part of this, I can construct the entities, feature views, feature services, etc, a instance of the ParsedRepo class for Feast. What I need is the ability to pass my instance into the apply_total method.

Describe the solution you’d like Remove the call to parse_repo from the apply_total ( link ) and instead perform that call elsewhere and pass the ParsedRepo object into the apply_total function.

Describe alternatives you’ve considered Copy the apply_total method into my code and make the changes needed. The reason why I don’t like this approach is because I would be maintaining a copy/paste of the method, while the change in the Feast repo wouldn’t impact functionality since feast apply would still call parse_repo and then pass the object to apply_total.

@cli.command("apply", cls=NoOptionDefaultFormat)
@click.option(
    "--skip-source-validation",
    is_flag=True,
    help="Don't validate the data sources by checking for that the tables exist.",
)
@click.pass_context
def apply_total_command(ctx: click.Context, skip_source_validation: bool):
    """
    Create or update a feature store deployment
    """
    repo = ctx.obj["CHDIR"]
    cli_check_repo(repo)
    repo_config = load_repo_config(repo)
    try:
        parsed_repo = parse_repo(repo) # <- this
        apply_total(parsed_repo, repo_config, repo, skip_source_validation)
    except FeastProviderLoginError as e:
        print(str(e))

[pyalex]

Hi @nossrannug , I think this apply_total was not intended to be used directly and seems to be written specifically for CLI. That being said, it could be easily split into sub-functions. And those sub-functions could be reused directly or new interfaces can be created. If you have time, please feel free to contribute.

[nossrannug]

So something like this?

@log_exceptions_and_usage
def apply_total(repo_config: RepoConfig, repo_path: Path, skip_source_validation: bool):

    os.chdir(repo_path)
    repo = _parse_repo(repo_path)
    do_stuff(repo_config, repo, sip_source_validation)

def do_stuff(repo_config: RepoConfig, repo: RepoContents, skip_source_validation: bool):
   project, registry, store = _prepare_registry(repo_config)

    if not skip_source_validation:
        data_sources = [t.batch_source for t in repo.feature_views]
        # Make sure the data source used by this feature view is supported by Feast
        for data_source in data_sources:
            data_source.validate(store.config)

    # For each object in the registry, determine whether it should be kept or deleted.
    (
        all_to_apply,
        all_to_delete,
        views_to_delete,
        views_to_keep,
    ) = extract_objects_for_apply_delete(project, registry, repo)

    diff = store.apply(all_to_apply, objects_to_delete=all_to_delete, partial=False)

    log_cli_output(diff, views_to_delete, views_to_keep)