Support for full-archive / academic research track endpoints

Hello 😃 this library has been great to use, it’s been incredibly useful with the research work I have been doing. Currently, if my understanding is correct this only supports the “standard” search endpoints search/tweets.json

After doing some digging I’ve found that this endpoint won’t offer the amount of data needed for the research I’m doing, thus I would need something like this endpoint for full-archive search which seems to only be offered under the v2 of the twitter API, /2/tweets/search/all (which is only for the academic research product track, interestingly)

I’m not sure if that is something this project supports, in the case that it’s not supported yet I’m assuming there’s a fair amount of technical debt in order to make this library compatible?

Since most of my research work depends on this Scala library, I’d be happy to contribute if need be 😃

[TheConner]

Update: there is a bug in my initial implementation, although figuring out why this is happening is a tad cryptic. Oddly enough all the tests pass, but when I use twitter4s within my application I get

Exception in thread "main" java.lang.AbstractMethodError: Receiver class com.danielasfregola.twitter4s.http.clients.rest.RestClient does not define or inherit an implementation of the resolved method 'abstract void de$heikoseeberger$akkahttpjson4s$Json4sSupport$_setter_$de$heikoseeberger$akkahttpjson4s$Json4sSupport$$jsonSourceStringMarshaller_$eq(akka.http.scaladsl.marshalling.Marshaller)' of interface de.heikoseeberger.akkahttpjson4s.Json4sSupport.
	at de.heikoseeberger.akkahttpjson4s.Json4sSupport.$init$(Json4sSupport.scala:96)
	at com.danielasfregola.twitter4s.http.clients.rest.RestClient.<init>(RestClient.scala:17)
	at com.danielasfregola.twitter4s.TwitterRestClient.<init>(TwitterRestClient.scala:41)
	at com.danielasfregola.twitter4s.TwitterRestClient$.apply(TwitterRestClient.scala:91)
	at com.danielasfregola.twitter4s.TwitterRestClient$.apply(TwitterRestClient.scala:75)
	at ca.advtech.ar2t.data.TweetIngest.<init>(TweetIngest.scala:28)
	at ca.advtech.ar2t.main$.main(main.scala:69)
	at ca.advtech.ar2t.TestRunMain$.main(TestRunMain.scala:5)
	at ca.advtech.ar2t.TestRunMain.main(TestRunMain.scala)

Continuing to investigate…

[TheConner]

It was due to a dependency issue, my application uses a different scala version (Apache spark is behind a few versions) so I had to build twitter4s for a different scala version & some of the dependency changes I made on my end didn’t work. Also, building twitter4s as a jar and importing that jar in meant that I had to manually include dependencies

Anywho, after that and some other dependency pains I managed to fix the issues in my implementation, so now I’m sucessfully using twitter4s using bearer-token auth with the full-archive search endpoint!

PR is made, let me know if there’s any changes you would like me to make to it 😃