Column based search is unimplemented in Atlas?

On amundsen advanced search, use a query term for column, and the xhr fails (that too silently without informing user)

Expected Behavior

Should list down all the column entries and/or all the tables with that column

Current Behavior

No results showup, api returns 500 Amundsen Search logs

    tables, approx_count = self._atlas_basic_search(query_params)
UnboundLocalError: local variable 'query_params' referenced before assignment

Possible Solution

Reason: Looking at those lines it seems team had no intension to build the feature. If that is the case then please document the known limitations. In that case we would not have chosen amundsen if known beforehand .

https://github.com/lyft/amundsensearchlibrary/blob/17c673903e2db3b1145af69fb31659d7be185eb4/search_service/proxy/atlas.py#L237-L240

Suggested solution (in case you change your mind to implement it): Use atlas basic search api POST /api/atlas/v2/search/basic with this payload

{
	"excludeDeletedEntities": true,
	"includeSubClassifications": true,
	"includeSubTypes": true,
	"includeClassificationAttributes": true,
	"entityFilters": null,
	"tagFilters": null,
	"attributes": [],
	"query": "😀 😀 Your fancy query string <3 ",
	"limit": 25,
	"offset": 0,
	"typeName": "Column",
	"classification": null,
	"termName": null
}

Steps to Reproduce

1. Open the advanced search page
2. Choose a query string form column field
3. Click on apply
4. Or just visit /search?resource=table&index=0&filters=%7B"column"%3A"your_fancy_query"%7D

Screenshots (if appropriate)

Context

A simple explorer software like apache superset or metabase supports searching tables with free test. No one will use a new software if it only reinvents the same feature. Column searching could be at least some feature to take a stand for amundsen.

Your Environment

Amunsen version used: dockerimages : amundsendev/amundsen-search:2.4.0 amundsendev/amundsen-metadata:2.5.4 amundsendev/amundsen-frontend-oidc:2.2.2

Deployment: k8s behind ingress

[mgorsk1]

Hey @pPanda-beta that’s correct, when implementing first version of advanced-search with Atlas we only took into account following attributes: database, table, cluster and tag (classification). This is because enabling advanced search by any other attribute mentioned by you (column name, schema) entails more complexity in implementation.

However, there was no decision not to support additional search filters, atlas proxy in amundsensearchlibrary can be extended with this support and we welcome any suggestion on how to approach this. Column search would, indeed, be very helpful. Using basic search for that would require solving issue of deduplication and accurate total_count rendering (if using basic search for this).

@feng-tao not sure if bug or actually a feature request ?

[mgorsk1]

With current atlas limitations in regard to search there is nothing more to be done on Amundsen side to improve this. If extended search capabilities are required from Atlas then the most efficient course of action is to bring it up with Atlas community directly.

Alternatively, you might want to try out atlas search extractor from databuilder to sync Atlas with ES. This solution will give you search capabilities on both column names and descriptions.