question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. ItΒ collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Undocumented dependencies in text classification example

See original GitHub issue

Text classification example depends on sentencepiece and subword_nmt packages. These should be documented in the README for the example.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
harshbafnacommented, Mar 16, 2020

@fbbradheintz This looks like some issue with the conda package for TorchText

conda doesn’t install sentencepiece as dependency for torchtext :

ubuntu@ip-172-31-22-66:~$ conda install torchtext -c pytorch
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.5.12
  latest version: 4.8.2

Please update conda by running

    $ conda update -n base -c defaults conda

## Package Plan ##

  environment location: /home/ubuntu/anaconda3

  added / updated specs: 
    - torchtext

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    torchtext-0.5.0            |             py_1         1.4 MB  pytorch

The following NEW packages will be INSTALLED:

    cudatoolkit: 10.1.243-h6bb024c_0                          
    ninja:       1.9.0-py36hfd86e86_0                         
    pytorch:     1.4.0-py3.6_cuda10.1.243_cudnn7.6.3_0 pytorch
    torchtext:   0.5.0-py_1                            pytorch
    tqdm:        4.42.1-py_0                                  

Proceed ([y]/n)? n

Doing pip show torchtext after installing torchtext with conda lists sentencepiece as requirement :

ubuntu@ip-172-31-22-66:~$ pip show torchtext
Name: torchtext
Version: 0.5.0
Summary: Text utilities and datasets for PyTorch
Home-page: https://github.com/pytorch/text
Author: PyTorch core devs and James Bradbury
Author-email: jekbradbury@gmail.com
License: BSD
Location: /home/ubuntu/anaconda3/lib/python3.6/site-packages
Requires: tqdm, requests, torch, numpy, six, sentencepiece
Required-by: 
ubuntu@ip-172-31-22-66:~$ 

Directly installing torchtext through pip command installs sentencepiece as dependency

ubuntu@ip-172-31-22-66:~$ pip install torchtext
Collecting torchtext
  Downloading https://files.pythonhosted.org/packages/79/ef/54b8da26f37787f5c670ae2199329e7dccf195c060b25628d99e587dac51/torchtext-0.5.0-py3-none-any.whl (73kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 81kB 8.3MB/s 
Requirement already satisfied: six in ./anaconda3/lib/python3.6/site-packages (from torchtext) (1.14.0)
Collecting torch
  Downloading https://files.pythonhosted.org/packages/24/19/4804aea17cd136f1705a5e98a00618cb8f6ccc375ad8bfa437408e09d058/torch-1.4.0-cp36-cp36m-manylinux1_x86_64.whl (753.4MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 753.4MB 6.1kB/s 
Collecting sentencepiece
  Downloading https://files.pythonhosted.org/packages/74/f4/2d5214cbf13d06e7cb2c20d84115ca25b53ea76fa1f0ade0e3c9749de214/sentencepiece-0.1.85-cp36-cp36m-manylinux1_x86_64.whl (1.0MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1.0MB 37.9MB/s 
Requirement already satisfied: requests in ./anaconda3/lib/python3.6/site-packages (from torchtext) (2.20.0)
Collecting tqdm
  Downloading https://files.pythonhosted.org/packages/47/55/fd9170ba08a1a64a18a7f8a18f088037316f2a41be04d2fe6ece5a653e8f/tqdm-4.43.0-py2.py3-none-any.whl (59kB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 61kB 152kB/s 
Requirement already satisfied: numpy in ./anaconda3/lib/python3.6/site-packages (from torchtext) (1.16.4)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./anaconda3/lib/python3.6/site-packages (from requests->torchtext) (3.0.4)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in ./anaconda3/lib/python3.6/site-packages (from requests->torchtext) (1.23)
Requirement already satisfied: idna<2.8,>=2.5 in ./anaconda3/lib/python3.6/site-packages (from requests->torchtext) (2.7)
Requirement already satisfied: certifi>=2017.4.17 in ./anaconda3/lib/python3.6/site-packages (from requests->torchtext) (2019.11.28)
Installing collected packages: torch, sentencepiece, tqdm, torchtext
Successfully installed sentencepiece-0.1.85 torch-1.4.0 torchtext-0.5.0 tqdm-4.43.0
0reactions
fbbradheintzcommented, Mar 18, 2020

FIxed in docs.

NB: It looks like sentencepiece is a dependency, but subword_nmt is not (at least in the latest version of torchtext).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Discovering Information Explaining API Types Using Text ...
classify tutorial sections as explaining a given API type or not with a precision between ... that take into account dependencies between words...
Read more >
Text Analysis using SVM - RPubs
SVM uses features of the the categories to predict its classification. In the case of text analysis, if the category uses certain words,Β ......
Read more >
Graph-based Representations for Text Classification
Below is an example of a dependency graph constructed from a random sentence from one of our corpora. Are you able to reconstruct...
Read more >
udpipe.pdf
a character vector: The character vector contains the text you want to tok- enize, lemmatise, tag and perform dependency parsing. The names ofΒ ......
Read more >
Past and future uses of text mining in ecology and evolution
Training a classifier requires examples of both positive and negative cases (e.g. relevant and irrelevant articles). Databases that reportΒ ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found