Optimize text_classifer model memory usage
See original GitHub issueRelated issue: https://github.com/brave/brave-browser/issues/21602
Parsed text classifier model (component id : emgmepnebbddgnkhfmhdhmjifkglkamo) takes ~24mb of memory in Ads Service process.
It’s looks excessive and we should find a way to make it smaller. Another problem is that peak usage on start is more than 200mb.
Here is the code that produce 24mb memory allocation: https://github.com/brave/brave-core/blob/master/vendor/bat-native-ads/src/bat/ads/internal/ml/pipeline/text_processing/text_processing.cc#L63
Issue Analytics
- State:
- Created a year ago
- Comments:8 (4 by maintainers)
Top Results From Across the Web
Text Classification: All Tips and Tricks from 5 Kaggle ...
Optimize the memory by reducing the size of some attributes · Use open-source libraries such as Dask to read and manipulate the data,...
Read more >Optimize PyTorch Performance for Speed and Memory ...
Overall, you can optimize the time and memory usage by 3 key points. First, reduce the i/o (input/output) as much as possible so...
Read more >Text Mining - Computation time and memory usage
Dear all, I am working on a text mining use case with a data set of around 80.000 data sets and 33 Attributes...
Read more >Expanded fastText library now fits on smaller-memory devices
fastText's models now fit on smartphones and small computers like Raspberry Pi devices thanks to a new functionality that reduces memory usage.
Read more >An Improved Text Classification Model Based on Memory ...
This paper proposes a text classification model, called improved memory neural network model, which is used to process large-scale training ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found

Makes sense, thanks for the clarification @atuchin-m 👍🏻
@LaurenWags @LaurenWags @btlechowski Big thanks for verification!
Just want to clarify one thing:
Memory footprinton Linux/Mac is counted in a different way that on Windows. That’s why it’s ok that absolute numbers are not match across OSes, so please don’t be confused by this.