Unable install scrapy on Ubuntu 16.04 with python 3+

Hello.

I tried to install Scrapy as in official manual [(http://doc.scrapy.org/en/master/topics/ubuntu.html#topics-ubuntu] on fresh installed Ubuntu 16.04 and it fails. Then I take few steps from [http://stackoverflow.com/questions/37669290/how-to-install-scrapy-on-unbuntu-16-04/37677910#37677910)] before process by manual:

apt-get install -y \
    python3 \
    python-dev \
    python3-dev

# for cryptography
apt-get install -y \
    build-essential \
    libssl-dev \
    libffi-dev

# for lxml
apt-get install -y \
    libxml2-dev \
    libxslt-dev

# install pip
apt-get install -y python-pip 

and this fails too. Here my console log:

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

p@ScrapyPython3:~$ apt-get install -y \
>     python3 \
>     python-dev \
>     python3-dev
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
p@ScrapyPython3:~$ sudo apt-get install -y     python3     python-dev     python3-dev
[sudo] password for p: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python3 is already the newest version (3.5.1-3).
The following additional packages will be installed:
  libexpat1-dev libpython-dev libpython2.7-dev libpython3-dev libpython3.5-dev
  python2.7-dev python3.5-dev
The following NEW packages will be installed:
  libexpat1-dev libpython-dev libpython2.7-dev libpython3-dev libpython3.5-dev
  python-dev python2.7-dev python3-dev python3.5-dev
0 upgraded, 9 newly installed, 0 to remove and 177 not upgraded.
Need to get 65.9 MB of archives.
After this operation, 95.9 MB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libexpat1-dev amd64 2.1.0-7ubuntu0.16.04.1 [115 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libpython2.7-dev amd64 2.7.11-7ubuntu1 [27.8 MB]
Get:3 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libpython-dev amd64 2.7.11-1 [7,728 B]
Get:4 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libpython3.5-dev amd64 3.5.1-10 [37.3 MB]
Get:5 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libpython3-dev amd64 3.5.1-3 [6,926 B]
Get:6 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 python2.7-dev amd64 2.7.11-7ubuntu1 [280 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 python-dev amd64 2.7.11-1 [1,160 B]
Get:8 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 python3.5-dev amd64 3.5.1-10 [413 kB]
Get:9 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 python3-dev amd64 3.5.1-3 [1,186 B]
Fetched 65.9 MB in 3min 38s (301 kB/s)                                         
Selecting previously unselected package libexpat1-dev:amd64.
(Reading database ... 205453 files and directories currently installed.)
Preparing to unpack .../libexpat1-dev_2.1.0-7ubuntu0.16.04.1_amd64.deb ...
Unpacking libexpat1-dev:amd64 (2.1.0-7ubuntu0.16.04.1) ...
Selecting previously unselected package libpython2.7-dev:amd64.
Preparing to unpack .../libpython2.7-dev_2.7.11-7ubuntu1_amd64.deb ...
Unpacking libpython2.7-dev:amd64 (2.7.11-7ubuntu1) ...
Selecting previously unselected package libpython-dev:amd64.
Preparing to unpack .../libpython-dev_2.7.11-1_amd64.deb ...
Unpacking libpython-dev:amd64 (2.7.11-1) ...
Selecting previously unselected package libpython3.5-dev:amd64.
Preparing to unpack .../libpython3.5-dev_3.5.1-10_amd64.deb ...
Unpacking libpython3.5-dev:amd64 (3.5.1-10) ...
Selecting previously unselected package libpython3-dev:amd64.
Preparing to unpack .../libpython3-dev_3.5.1-3_amd64.deb ...
Unpacking libpython3-dev:amd64 (3.5.1-3) ...
Selecting previously unselected package python2.7-dev.
Preparing to unpack .../python2.7-dev_2.7.11-7ubuntu1_amd64.deb ...
Unpacking python2.7-dev (2.7.11-7ubuntu1) ...
Selecting previously unselected package python-dev.
Preparing to unpack .../python-dev_2.7.11-1_amd64.deb ...
Unpacking python-dev (2.7.11-1) ...
Selecting previously unselected package python3.5-dev.
Preparing to unpack .../python3.5-dev_3.5.1-10_amd64.deb ...
Unpacking python3.5-dev (3.5.1-10) ...
Selecting previously unselected package python3-dev.
Preparing to unpack .../python3-dev_3.5.1-3_amd64.deb ...
Unpacking python3-dev (3.5.1-3) ...
Processing triggers for doc-base (0.10.7) ...
Processing 1 added doc-base file...
Processing triggers for man-db (2.7.5-1) ...
Setting up libexpat1-dev:amd64 (2.1.0-7ubuntu0.16.04.1) ...
Setting up libpython2.7-dev:amd64 (2.7.11-7ubuntu1) ...
Setting up libpython-dev:amd64 (2.7.11-1) ...
Setting up libpython3.5-dev:amd64 (3.5.1-10) ...
Setting up libpython3-dev:amd64 (3.5.1-3) ...
Setting up python2.7-dev (2.7.11-7ubuntu1) ...
Setting up python-dev (2.7.11-1) ...
Setting up python3.5-dev (3.5.1-10) ...
Setting up python3-dev (3.5.1-3) ...
p@ScrapyPython3:~$ apt-get install -y \
>     build-essential \
>     libssl-dev \
>     libffi-dev
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
p@ScrapyPython3:~$ sudo apt-get install -y     build-essential     libssl-dev     libffi-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
build-essential is already the newest version (12.1ubuntu2).
build-essential set to manually installed.
The following additional packages will be installed:
  libssl-doc zlib1g-dev
The following NEW packages will be installed:
  libffi-dev libssl-dev libssl-doc zlib1g-dev
0 upgraded, 4 newly installed, 0 to remove and 177 not upgraded.
Need to get 2,801 kB of archives.
After this operation, 11.1 MB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 zlib1g-dev amd64 1:1.2.8.dfsg-2ubuntu4 [168 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libssl-dev amd64 1.0.2g-1ubuntu4.1 [1,394 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libssl-doc all 1.0.2g-1ubuntu4.1 [1,078 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libffi-dev amd64 3.2.1-4 [161 kB]
Fetched 2,801 kB in 14s (190 kB/s)                                             
Selecting previously unselected package zlib1g-dev:amd64.
(Reading database ... 205764 files and directories currently installed.)
Preparing to unpack .../zlib1g-dev_1%3a1.2.8.dfsg-2ubuntu4_amd64.deb ...
Unpacking zlib1g-dev:amd64 (1:1.2.8.dfsg-2ubuntu4) ...
Selecting previously unselected package libssl-dev:amd64.
Preparing to unpack .../libssl-dev_1.0.2g-1ubuntu4.1_amd64.deb ...
Unpacking libssl-dev:amd64 (1.0.2g-1ubuntu4.1) ...
Selecting previously unselected package libssl-doc.
Preparing to unpack .../libssl-doc_1.0.2g-1ubuntu4.1_all.deb ...
Unpacking libssl-doc (1.0.2g-1ubuntu4.1) ...
Selecting previously unselected package libffi-dev:amd64.
Preparing to unpack .../libffi-dev_3.2.1-4_amd64.deb ...
Unpacking libffi-dev:amd64 (3.2.1-4) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for doc-base (0.10.7) ...
Processing 1 added doc-base file...
Processing triggers for install-info (6.1.0.dfsg.1-5) ...
Setting up zlib1g-dev:amd64 (1:1.2.8.dfsg-2ubuntu4) ...
Setting up libssl-dev:amd64 (1.0.2g-1ubuntu4.1) ...
Setting up libssl-doc (1.0.2g-1ubuntu4.1) ...
Setting up libffi-dev:amd64 (3.2.1-4) ...
p@ScrapyPython3:~$ apt-get install -y \
>     libxml2-dev \
>     libxslt-dev
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
p@ScrapyPython3:~$ sudo apt-get install -y     libxml2-dev     libxslt-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libxslt1-dev' instead of 'libxslt-dev'
The following additional packages will be installed:
  icu-devtools libicu-dev
Suggested packages:
  icu-doc
The following NEW packages will be installed:
  icu-devtools libicu-dev libxml2-dev libxslt1-dev
0 upgraded, 4 newly installed, 0 to remove and 177 not upgraded.
Need to get 9,860 kB of archives.
After this operation, 47.4 MB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 icu-devtools amd64 55.1-7 [165 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libicu-dev amd64 55.1-7 [8,546 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 libxml2-dev amd64 2.9.3+dfsg1-1ubuntu0.1 [743 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libxslt1-dev amd64 1.1.28-2.1 [406 kB]
Fetched 9,860 kB in 36s (274 kB/s)                                             
Selecting previously unselected package icu-devtools.
(Reading database ... 207591 files and directories currently installed.)
Preparing to unpack .../icu-devtools_55.1-7_amd64.deb ...
Unpacking icu-devtools (55.1-7) ...
Selecting previously unselected package libicu-dev:amd64.
Preparing to unpack .../libicu-dev_55.1-7_amd64.deb ...
Unpacking libicu-dev:amd64 (55.1-7) ...
Selecting previously unselected package libxml2-dev:amd64.
Preparing to unpack .../libxml2-dev_2.9.3+dfsg1-1ubuntu0.1_amd64.deb ...
Unpacking libxml2-dev:amd64 (2.9.3+dfsg1-1ubuntu0.1) ...
Selecting previously unselected package libxslt1-dev:amd64.
Preparing to unpack .../libxslt1-dev_1.1.28-2.1_amd64.deb ...
Unpacking libxslt1-dev:amd64 (1.1.28-2.1) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for doc-base (0.10.7) ...
Processing 1 added doc-base file...
Setting up icu-devtools (55.1-7) ...
Setting up libicu-dev:amd64 (55.1-7) ...
Setting up libxml2-dev:amd64 (2.9.3+dfsg1-1ubuntu0.1) ...
Setting up libxslt1-dev:amd64 (1.1.28-2.1) ...
p@ScrapyPython3:~$ apt-get install -y python-pip
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
p@ScrapyPython3:~$ sudo apt-get install -y python-pip
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libpython-all-dev python-all python-all-dev python-pip-whl
  python-pkg-resources python-setuptools python-wheel
Suggested packages:
  python-setuptools-doc
The following NEW packages will be installed:
  libpython-all-dev python-all python-all-dev python-pip python-pip-whl
  python-pkg-resources python-setuptools python-wheel
0 upgraded, 8 newly installed, 0 to remove and 177 not upgraded.
Need to get 1,583 kB of archives.
After this operation, 3,065 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 libpython-all-dev amd64 2.7.11-1 [992 B]
Get:2 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 python-all amd64 2.7.11-1 [978 B]
Get:3 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 python-all-dev amd64 2.7.11-1 [1,000 B]
Get:4 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 python-pip-whl all 8.1.1-2ubuntu0.1 [1,110 kB]
Get:5 http://us.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 python-pip all 8.1.1-2ubuntu0.1 [144 kB]
Get:6 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 python-pkg-resources all 20.7.0-1 [108 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 python-setuptools all 20.7.0-1 [169 kB]
Get:8 http://us.archive.ubuntu.com/ubuntu xenial/universe amd64 python-wheel all 0.29.0-1 [48.0 kB]
Fetched 1,583 kB in 3s (422 kB/s)  
Selecting previously unselected package libpython-all-dev:amd64.
(Reading database ... 208343 files and directories currently installed.)
Preparing to unpack .../libpython-all-dev_2.7.11-1_amd64.deb ...
Unpacking libpython-all-dev:amd64 (2.7.11-1) ...
Selecting previously unselected package python-all.
Preparing to unpack .../python-all_2.7.11-1_amd64.deb ...
Unpacking python-all (2.7.11-1) ...
Selecting previously unselected package python-all-dev.
Preparing to unpack .../python-all-dev_2.7.11-1_amd64.deb ...
Unpacking python-all-dev (2.7.11-1) ...
Selecting previously unselected package python-pip-whl.
Preparing to unpack .../python-pip-whl_8.1.1-2ubuntu0.1_all.deb ...
Unpacking python-pip-whl (8.1.1-2ubuntu0.1) ...
Selecting previously unselected package python-pip.
Preparing to unpack .../python-pip_8.1.1-2ubuntu0.1_all.deb ...
Unpacking python-pip (8.1.1-2ubuntu0.1) ...
Selecting previously unselected package python-pkg-resources.
Preparing to unpack .../python-pkg-resources_20.7.0-1_all.deb ...
Unpacking python-pkg-resources (20.7.0-1) ...
Selecting previously unselected package python-setuptools.
Preparing to unpack .../python-setuptools_20.7.0-1_all.deb ...
Unpacking python-setuptools (20.7.0-1) ...
Selecting previously unselected package python-wheel.
Preparing to unpack .../python-wheel_0.29.0-1_all.deb ...
Unpacking python-wheel (0.29.0-1) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libpython-all-dev:amd64 (2.7.11-1) ...
Setting up python-all (2.7.11-1) ...
Setting up python-all-dev (2.7.11-1) ...
Setting up python-pip-whl (8.1.1-2ubuntu0.1) ...
Setting up python-pip (8.1.1-2ubuntu0.1) ...
Setting up python-pkg-resources (20.7.0-1) ...
Setting up python-setuptools (20.7.0-1) ...
Setting up python-wheel (0.29.0-1) ...
p@ScrapyPython3:~$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 627220E7
Executing: /tmp/tmp.ZQw2BsNciq/gpg.1.sh --keyserver
hkp://keyserver.ubuntu.com:80
--recv
627220E7
gpg: requesting key 627220E7 from hkp server keyserver.ubuntu.com
gpg: key 627220E7: public key "Scrapy Team (APT Signing Key) <[e-mail]>" imported
gpg: Total number processed: 1
gpg:               imported: 1  (RSA: 1)
p@ScrapyPython3:~$ echo 'deb http://archive.scrapy.org/ubuntu scrapy main' | sudo tee /etc/apt/sources.list.d/scrapy.list
deb http://archive.scrapy.org/ubuntu scrapy main
p@ScrapyPython3:~$ 
p@ScrapyPython3:~$ sudo apt-get update && sudo apt-get install scrapy
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
Hit:2 http://us.archive.ubuntu.com/ubuntu xenial InRelease                     
Get:3 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease [94.5 kB]   
Get:4 http://archive.scrapy.org/ubuntu scrapy InRelease [6,002 B]              
Hit:5 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease           
Get:6 http://archive.scrapy.org/ubuntu scrapy/main amd64 Packages [6,085 B]
Get:7 http://archive.scrapy.org/ubuntu scrapy/main i386 Packages [6,085 B]
Fetched 207 kB in 1s (161 kB/s)     
Reading package lists... Done
W: http://archive.scrapy.org/ubuntu/dists/scrapy/InRelease: Signature by key 65D1B1CE9BCE5D04D51464F58F62CB1F627220E7 uses weak digest algorithm (SHA1)
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 scrapy : Depends: python-support (>= 0.90.0) but it is not installable
E: Unable to correct problems, you have held broken packages.
p@ScrapyPython3:~$ 

[redapple]

@pavelmorozov , here’s a example installing Scrapy on Ubuntu 16:04, with pip, using Python 2, and where $ scrapy version fails, and where you can use $ python -m scrapy.cmdline version https://gist.github.com/redapple/a3323f13a83f5630f2003f964d4432c5 (I’m using Docker to work with a “clean” Ubuntu 16:04) I’ll prepare Python 3 examples

[Draidel]

I am having the same issue. 😦