question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Solr arrays not getting inserted in to solr

See original GitHub issue

Hi I’m running mongo-connector latest build and I’m unable to insert list values into solr multiValued fields. In my test index, I have the followling solr fields.

<field name="_id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
   <field name="_ts" type="long" indexed="true" stored="true" multiValued="false" />
   <field name="_ns" type="string" indexed="true" stored="true" multiValued="false" />
   <field name="spec_name" type = "string" indexed = "true" stored = "true" multiValued="true"/>
   <field name="spec_value" type = "string" indexed = "true" stored = "true" multiValued="true"/>

Given these two documents from my mongodb

{ "_id" : ObjectId("5320d76ebdca98e379b97b9b"), "jobID" : "2597402", "spec_value" : "20" }
{ "_id" : ObjectId("5320d7ccbdca98e379b97b9c"), "jobID" : "2597402", "spec_name" : [  "one",  "two" ], "spec_value" : [  "20",  "test" ] }

the first document inserts the spec_value field correctly, but the second does not.

Is this possibly a bug?

Also, here is my mongo-connector call for starting it up.

mongo-connector -m localhost:27017 -t http://localhost:8181/solr/content  -d mongo-connector/mongo_connector/doc_managers/solr_doc_manager.py -n test.test -a tomcat -p tomcat -i spec_name,spec_value,jobID

Thanks for any help you may offer, Tim

Issue Analytics

  • State:closed
  • Created 10 years ago
  • Comments:5

github_iconTop GitHub Comments

1reaction
llvttcommented, Mar 12, 2014

@acmeinternetsolutions , Mongo Connector unwinds arrays, so

{ "_id" : ObjectId("5320d7ccbdca98e379b97b9c"), 
  "jobID" : "2597402", 
  "spec_name" : [  "one",  "two" ], 
  "spec_value" : [  "20",  "test" ] }

will be turned into the following before insertion into Solr:

{ "_id" : ObjectId("5320d7ccbdca98e379b97b9c"), 
  "jobID" : "2597402", 
  "spec_name.0" : "one", 
  "spec_name.1": "two", 
  "spec_value.0" : "20", 
  "spec_value.1": "test" }

This is necessary if there is a sub-document inside of an array. Solr has no notion of “sub-documents” like MongoDB does, and so these need to be flattened, and in order to do this, arrays must also be unwound. For a lengthier discussion about this, you can refer to #89. You might be able to get around this with the copyField directive in your schema.xml.

0reactions
pksomancommented, Jan 13, 2016

I see this post is closed. However, I would like to know if there is any way to handle an array of String within an array of Object.

Ex:

{ “name”: “doc1”, “a” : [ { “b” : “something.0”, “c” : [ “1001”, “1002” ] },{ “b” : “something.1”, “c” : [ “1006”, “1007” ] } ] }, { “name”: “doc2”, “a” : [ { “b” : “something.2”, “c” : [ “1001”, “1002” ] },{ “b” : “something 3”, “c” : [ “1006”, “1007” ] } ] }

This get converted as { “name” : “doc1”, a.0.b = “something.0”, a.0.c.0 = “1001”, a.0.c.1 = “1002” a.1.b=“something.1”, a.1.c.0 = “1006”, a.1.c.1 = “1007” }, { “name” : “doc2”, a.0.b = “something.2”, a.0.c.0 = “1001”, a.0.c.1 = “1002”, a.1.b=“something.3”, a.1.c.0 = “1006”, a.1.c.1 = “1007” }

Now how can I search in Solr to match all documents that has a..b=“something.0” and a..c contains “1002”

Sorry for the lengthy detail.

Read more comments on GitHub >

github_iconTop Results From Across the Web

How to get solr result's doc fields in str rather than arr?
It took some time, but the best way I found to fix all of my fields was insert a JSON record and check...
Read more >
Uploading Data with Index Handlers - Apache Solr
Before a commit has been issued, newly indexed content is not visible to searches. The commit operation opens a new searcher, and triggers...
Read more >
Transforming Result Documents - Apache Solr
This transformer returns all descendant documents of each parent document matching your query in a flat list nested inside the matching parent document....
Read more >
Major Changes in Solr 8 | Apache Solr Reference Guide 8.0
Solr now has the ability to store information about document relationships in the index. This stored information can be used for queries. The...
Read more >
Response Writers | Apache Solr Reference Guide 6.6
If the indent parameter is used, and has a non-blank value, then Solr will make some attempts at indenting its XML response to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found