Tech-radar support for CSV data source

Now there is a simple way how to feed data from tech-radar plugin via “getData” attribute. We did implement our own feeder that loads data from CSV. However when we tried to put the url in the configuration we hit the wall many times as there is no easy way, how to achieve this “out” of the component.

I have prepared an enhancemen that fullfills our needs of loading tech-radar data from CSV. I would like to contribute to the comunity and want to gather feedback to this feature.

Feature Suggestion

New attribute useCsvLoader in the TechRadarPage component:

<TechRadarPage ... useCsvLoader />

New configuration:

techradar:
  csvEntriesUrl: https://location-of-csv-source/xxx.csv
  quadrants:
    - id: lngfwk
      name: Languages & Frameworks
    - id: tools
      name: Tools
   ...
  rings:
    - id: adopt
      name: Adopt
      color: '#009688'
    - id: trial
      name: Trial
      color: '#6acbef'
   ...

Possible Implementation

plugins/tech-radar/src/components/RadarComponent.tsx

...
import { TechRadarCsvDataLoader } from './csv-support/TechRadarCsvDataLoader';
...
const useTechRadarLoader = (props: TechRadarComponentProps) => {
  ...
  const configApi = useApi(configApiRef);
  const csvEntriesUrl = configApi.getOptionalString('techradar.csvEntriesUrl') ?? 'https://unknown-url-please-configure';
  ...
  const state = useAsync(async () => {
    if (useCsvLoader) {
      const response: TechRadarLoaderResponse = await TechRadarCsvDataLoader(csvEntriesUrl);
      return response;
    }
    if (getData) {
      ...
    }
    return undefined;
  }, [getData, errorApi]);
  ...
};

plugins/tech-radar/src/components/csv-support/TechRadarCsvDataLoader.ts

export { TechRadarCsvDataLoader };

import { GetRadarEntries } from "./GetRadarEntries";
import { TechRadarLoaderResponse } from "../../api";

const TechRadarCsvDataLoader = async (csvEntriesUrl: string) : Promise<TechRadarLoaderResponse> => {
    const entries = await GetRadarEntries(csvEntriesUrl);
    return Promise.resolve({
        quadrants: //TBD: load from config or get from parameter,
        rings: //TBD
        entries: entries,
    });
}

plugins/tech-radar/src/components/csv-support/GetRadarEntries.ts

import { RadarEntry } from "../../api";
import { GetCsvEntries } from "./GetCsvEntries";

export async function GetRadarEntries(
    csvEntriesUrl: string
): Promise<RadarEntry[]> {
    const entries: RadarEntry[] = [];
    const csvRecords = await GetCsvEntries(csvEntriesUrl);
    var i = 0;
    csvRecords.forEach((row: { name: string; ring: string; quadrant: string; movedState?: number; description: string; url?: string;}) => {
        i++;

        entries.push({
            url: row.url ?? '#',
            key: 'key_' + i,
            id: 'id_' + i,
            title: row.name,
            quadrant: row.quadrant,
            description: row.description,
            timeline: [
                {
                    //TBD: only one row used?
                    moved: row.movedState,
                    ringId: row.ring,
                    date: new Date('2020-01-01'), //TODO
                    description: row.description,
                }
            ]
        })
    });
    return Promise.resolve(entries);
}

plugins/tech-radar/src/components/csv-support/GetCsvEntries.ts

import parse from 'csv-parse/lib/sync';

export async function GetCsvEntries(
    csvEntriesUrl: string,
): Promise<any> {
    console.log("Fetching csv from %s", csvEntriesUrl);
    const response = await fetchUrl(csvEntriesUrl);
    const records = parse(response, {
        columns: true,
        skip_empty_lines: true,
        delimiter: ",",
    })
    return Promise.resolve(records);
}

const fetchUrl = async (
    url: string,
): Promise<string> => {
    const response = await fetch(url);
    if (response.status === 200)
        return await response.text() as string;
    throw new Error("Response not 200");
};

sample data (plugins/tech-radar/sample-data.csv):

name,ring,quadrant,movedState,description,area,url
Backstage,adopt,platforms,0,"An open platform for building developer portals: Powered by a centralized service catalog, Backstage restores order to your infrastructure and enables your product teams to ship high-quality code quickly — without compromising autonomy.",development,https://backstage.io/

Context

We want to load data for tech-radar from csv.

[benjdlambert]

Hey 👋 So i’ve been hacking a little on making the TechRadar a little more flexible on how you load data.

It’s not yet finished as I need to write some documentation, but in theory you can create your own class which implements TechRadarApi with a load method that should return the correct JSON object, and how you build that JSON object is entirely up to you. So you can read a CSV URL from config if you want and parse that CSV file and create the required JSON object.

#5891

[Fox32]

Actually that shouldn’t have a performance impact. With API I mean an interface in the frontend that an adopter can replace with an own implementation. See the example for the explore plugin linked above.