Media APIs: Attachement/Upload Problems

I’m not sure if this is an issue related to my general lack of js knowledge, or if it’s related to Apostrophe is self. So feel free to remove this if it’s not appropriate for this forum.

What I’m trying to do: After my piece is saved I’m trying to fetch some kml/xml data from another domain, create an apostrophe ‘attachment,’ and assign it to the piece that was just saved.

The problem: I’m stuck with uploading to Apostrophe. No matter what I pass as a “file” I end up with the error:

invalid: invalid
    at Object.error (/usr/local/www/apos-map/tracking/node_modules/apostrophe/modules/@apostrophecms/error/index.js:29:23)
    at post.upload (/usr/local/www/apos-map/tracking/node_modules/apostrophe/modules/@apostrophecms/attachment/index.js:153:33)
    at Array.<anonymous> (/usr/local/www/apos-map/tracking/node_modules/apostrophe/modules/@apostrophecms/module/index.js:169:36)
    at next (/usr/local/www/apos-map/tracking/node_modules/apostrophe/modules/@apostrophecms/module/index.js:112:31)
    at Array.<anonymous> (/usr/local/www/apos-map/tracking/node_modules/connect-multiparty/index.js:101:9)
    at listener (/usr/local/www/apos-map/tracking/node_modules/on-finished/index.js:169:15)
    at onFinish (/usr/local/www/apos-map/tracking/node_modules/on-finished/index.js:100:5)
    at callback (/usr/local/www/apos-map/tracking/node_modules/ee-first/index.js:55:10)
    at IncomingMessage.onevent (/usr/local/www/apos-map/tracking/node_modules/ee-first/index.js:93:5)
    at IncomingMessage.emit (events.js:412:35)
    at endReadableNT (internal/streams/readable.js:1334:12)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)

More Information I’ve tried passing the kml text I receive in as the file, saving it to a temp file on the server and passing the path, and finally below, creating a readStream of the temp file I saved to the server.

I’m not expecting someone here to write the code for me, but I’m out of things to google. I can’t figure out what the api needs to consider what I pass in a “file.”

As I stated at the beginning, I’m new to this, so perhaps my approach is all wrong. Is formData only appropriate when combined with an actual front-end form?

My Code

handlers(self) {
return {
beforeSave: {
    async getGarminKML(req, piece) {
    if (piece.aposMode == "published"){
    var fetch = require('node-fetch');
    var FormData = require('form-data');
    var fs = require('fs');
					
    inReachBaseURL = "https://share.garmin.com/Feed/Share/****redacted****?d1=";
    let beginUTC = new Date(piece.trackDateStart+"T"+piece.trackTimeStart).toISOString();
    let endUTC = new Date(piece.trackDateEnd+"T"+piece.trackTimeEnd).toISOString();
    kmlReqURL = inReachBaseURL.concat(beginUTC,"&d2=",endUTC);
		
    async function fetchAsync (url) {
        let response = await fetch(url);
        let data = await response.text();
        return data;
    }
					
    try {
        kml = await fetchAsync(kmlReqURL);
        const formData = new FormData();
		
        fs.writeFile('public/uploads/kmlTempFile.kml', kml , function (err) {
        if (err) throw err;
        console.log('Saved!');
        });
						
        const kmlTempFile = fs.createReadStream('public/uploads/kmlTempFile.kml');				
			
        formData.append('file',kmlTempFile);
        const attachment = await fetch('https://****redacted****/api/v1/@apostrophecms/attachment/upload', {
        method: 'POST',
        headers: {
        'Content-Type': 'multipart/form-data',
         'Authorization': 'ApiKey ****redacted****'
        },
        body: formData
        });
        }		
        catch (error){console.log("Error: ", error);}
        }
        },
        }
        }
        }

[boutell]

The first thing I notice is that you’re already on the server, yet you’re reaching out to the REST API. That is inefficient and involves passing api keys and so on. You should look at using the “apos.attachment.insert” method on the server side.

Second, kml is probably just not configured as a permitted extension for file uploads. That can be addressed through configuration of the @apostrophecms/attachment module’s “fileGroups” option. I would add it to the “office” group. Unfortunately for now at least you’d need to restate the entire option when replacing the default setting, which you can find in the @apostrophecms/attachment/index.js source code.

On Wed, Dec 1, 2021 at 11:30 AM Alex Bea @.***> wrote:

I’m wondering if the file path needs to be more specific. Nothing else is sticking out to me yet. I’m not an expert with streams, though. This looks similar to the core test of the upload route: https://github.com/apostrophecms/apostrophe/blob/main/test/pieces.js#L1239

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/apostrophecms/apostrophe/issues/3568#issuecomment-983813833, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAH27PA4DC5MONAJQV5X2LUOZERJANCNFSM5JC7VSXA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

–

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[boutell]

Where you wrote “my ignorance” we should read “our still-improving documentation regarding.” (:

On Sat, Dec 4, 2021 at 10:31 AM mporembs @.***> wrote:

I never got the attachment to work on the backend, as I originally had intended.

I did end up accomplishing my overall goals, though. For the piece type where I want to store the data permanently, I put the response into a text field and am processing it on the client side. For the piece type where I just want the latest data, I setup a cors-anywhere server and used that to access the data on the client side.

At any rate, I really appreciate the willingness of this forum to help. Despite my questions and methods revealing my ignorance to a lot of basic concepts.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/apostrophecms/apostrophe/issues/3568#issuecomment-986045801, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAH27OVTFBCBYT3HKNXI63UPIX6NANCNFSM5JC7VSXA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

–

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his