Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support for full-archive / academic research track endpoints

See original GitHub issue

Hello 😃 this library has been great to use, it’s been incredibly useful with the research work I have been doing. Currently, if my understanding is correct this only supports the “standard” search endpoints search/tweets.json

After doing some digging I’ve found that this endpoint won’t offer the amount of data needed for the research I’m doing, thus I would need something like this endpoint for full-archive search which seems to only be offered under the v2 of the twitter API, /2/tweets/search/all (which is only for the academic research product track, interestingly)

I’m not sure if that is something this project supports, in the case that it’s not supported yet I’m assuming there’s a fair amount of technical debt in order to make this library compatible?

Since most of my research work depends on this Scala library, I’d be happy to contribute if need be 😃

Issue Analytics

  • State:open
  • Created 3 years ago
  • Comments:11 (5 by maintainers)

github_iconTop GitHub Comments

TheConnercommented, Feb 20, 2021

Update: there is a bug in my initial implementation, although figuring out why this is happening is a tad cryptic. Oddly enough all the tests pass, but when I use twitter4s within my application I get

Exception in thread "main" java.lang.AbstractMethodError: Receiver class does not define or inherit an implementation of the resolved method 'abstract void de$heikoseeberger$akkahttpjson4s$Json4sSupport$_setter_$de$heikoseeberger$akkahttpjson4s$Json4sSupport$$jsonSourceStringMarshaller_$eq(akka.http.scaladsl.marshalling.Marshaller)' of interface de.heikoseeberger.akkahttpjson4s.Json4sSupport.
	at de.heikoseeberger.akkahttpjson4s.Json4sSupport.$init$(Json4sSupport.scala:96)
	at com.danielasfregola.twitter4s.TwitterRestClient.<init>(TwitterRestClient.scala:41)
	at com.danielasfregola.twitter4s.TwitterRestClient$.apply(TwitterRestClient.scala:91)
	at com.danielasfregola.twitter4s.TwitterRestClient$.apply(TwitterRestClient.scala:75)
	at ca.advtech.ar2t.main$.main(main.scala:69)
	at ca.advtech.ar2t.TestRunMain$.main(TestRunMain.scala:5)
	at ca.advtech.ar2t.TestRunMain.main(TestRunMain.scala)

Continuing to investigate…

TheConnercommented, Feb 23, 2021

It was due to a dependency issue, my application uses a different scala version (Apache spark is behind a few versions) so I had to build twitter4s for a different scala version & some of the dependency changes I made on my end didn’t work. Also, building twitter4s as a jar and importing that jar in meant that I had to manually include dependencies

Anywho, after that and some other dependency pains I managed to fix the issues in my implementation, so now I’m sucessfully using twitter4s using bearer-token auth with the full-archive search endpoint!

PR is made, let me know if there’s any changes you would like me to make to it 😃

Read more comments on GitHub >

github_iconTop Results From Across the Web

Academic research product details | Twitter Developer Platform
Academic Research product track. Full-archive search. Search the complete history of public Tweets based on your filters. Details. This endpoint lets you ...
Read more >
Introducing the new Academic Research product track
Now the Academic Research product track also includes access to the full-archive search endpoint, letting you get public Tweets from the ...
Read more >
Using the Twitter API for Academic Research Track - YouTube
I show how to use tweepy to search and download data from the Twitter v2 API, using tweepy.The notebook shown in the video...
Read more >
Twitter's new API platform now opened to academic researchers
In addition to gaining access to all the Twitter API v2 endpoints ... The Basic level of access on the Academic Research track...
Read more >
Getting historical Tweets using the full-archive search endpoint
Once you are approved to use the Academic Research product track, ... of operators that are supported in the full-archive search endpoint.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found