您的位置:永利集团登录网址 > 网络安全 > Installing Pycharm, Python Tensorflow, Cuda and cudnn in Ubu

Installing Pycharm, Python Tensorflow, Cuda and cudnn in Ubu

2019-10-05 01:45

星期日, 02. 九月 2018 11:58下午 - beautifulzzzz

The version compatibility across the OS and these packages is a nightmare for every new person who tries to use Tensorflow.  In here, I record the successful procedure to install everything listed in the title of this note.

ConnMan

   Connection Manager简称connman,connman是使用d-bus做为进程间通信机制来管理Linux网络链接的一种软件。在connman的d-bus接口中,有一个接口叫做service。该接口的全称为:net.connman.Service,自然是挂在net.connman上面的。做为net.connman.Service接口,它可以访问到network的细节以及用户偏好使用什么类型的网络(比如3g、wifi、wimax等等)。外部的程序就通过net.connman.Service的一堆Properties和Methods来实现它基本的功能。(可能诠释的不对,可能是整体的connman的功能)
        核心的逻辑如下:connman维护一个信号标记并且将所有可能连接上的网络服务建成一个列表(分类成普通网络、用户喜欢的以及上次使用的网络)。做为网络服务,它可以是以太网设备,WIFI,WIMAX,或者是远程蓝牙设备(比如手机)。

        这个列表的分类和排序是由connman内部实现,没必要由使用它的用户来决定怎么去将网络排序之类的。用户的决定(比如链接WIFI、关闭以太网等等)会通过connman实现,与此同时connman会重新对列表中的网络服务进行排序。

Related articles

  • Network configuration
  • Wireless network configuration
  • Category:Network configuration

ConnMan is a command-line network manager designed for use with embedded devices and fast resolve times. It is modular through a plugin architecture, but has native DHCP and NTP support.

图片 1

First of all, make sure that you have the exact same version of software in every steps, or it may not work.  To install Tensorflow at Ubuntu, you will need to install Ubuntu 16.04.  Run 

Contents

  • 1 Installation
    • 1.1 Desktop clients
  • 2 Usage
    • 2.1 Wired
    • 2.2 Wi-Fi
      • 2.2.1 Enabling and disabling wifi
      • 2.2.2 Connecting to an open access point
      • 2.2.3 Connecting to a protected access point
    • 2.3 Settings
    • 2.4 Technologies
  • 3 Tips and tricks
    • 3.1 Avoid changing the hostname
    • 3.2 Prefer ethernet to wireless
    • 3.3 Exclusive connection
    • 3.4 Connecting to eduroam (802.1X)
    • 3.5 Avoiding conflicts with local DNS server
    • 3.6 Blacklist interfaces
  • 4 Troubleshooting
    • 4.1 Error /net/connman/technology/wifi: Not supported
    • 4.2 Error /net/connman/technology/wifi: No carrier
    • 4.3 Error Failed to set hostname/domainname
    • 4.4 Unknown route on connection
  • 5 See also

1. Introduction

Bluez is the default Bluetooth protocol stack on Linux. It should be present and installed on your Linux distribution. If not, building and installing from source is not too difficult:

  • Download the latest stable source release of Bluez from here. Unzip the compressed file you downloaded.
  • Install the headers and libraries required for Bluez compilation:

图片 2

lsb_release -a 

Installation

Install the connman package. wpa_supplicant and bluez are optional dependencies required for Wi-Fi and Bluetooth functionality respectively.

Before enabling connman.service, ensure any existing network configuration is disabled.

2. Download And Install

I follow the blog (Installing Bluez 5.44 onto Raspbian?) to install bluez-5.50.

Download the most recent version from the official page:.

For example, at the time of writing it was 5.50, so I used(on my pi):

wget http://www.kernel.org/pub/linux/bluetooth/bluez-5.50.tar.xz

Then I extracted it and built it:

tar -xf bluez-5.50.tar.xz
cd bluez-5.50

Read the README! It lists the dependencies and the configure switches:

Install the dependencies first: (glib, dbus, libdbus, udev, etc.)

sudo apt install libdbus-1-dev libudev-dev libical-dev libreadline-dev

note: If you do not install the libdbus-1-dev, you will later get this strange error:

configure: error: D-Bus >= 1.6 is required

once you've installed dependencies, you can configure switches:

./configure --prefix=/usr --mandir=/usr/share/man --sysconfdir=/etc --localstatedir=/var  --enable-experimental

then do:

make
sudo make install

It takes maybe 10 minutes to compile. After installing, you should find bluetoothd in /usr/libexec/bluetooth. You should also see bluetoothd in /usr/lib/bluetooth.

Go to each of these directories and type

./bluetoothd --version

You'll note that the one in libexec is new and the one in lib is old.

In order to make sure that d-bus is talking to you new BlueZ 5.50 and not your old BlueZ 5.43, you need to tell systemd to use the new bluetooth daemon:

sudo vim /lib/systemd/system/bluetooth.service

Make sure the exec.start line points to your new daemon in /usr/libexec/bluetooth.

For me, that wasn't enough. No matter what, upon restart I always got bluetoothd 5.43... So I just created a symlink from the old one to the new.

First rename the old file:

sudo mv /usr/lib/bluetooth/bluetoothd /usr/lib/bluetooth/bluetoothd-543.orig

Create the symlink:

sudo ln -s /usr/libexec/bluetooth/bluetoothd /usr/lib/bluetooth/bluetoothd
sudo systemctl daemon-reload

That should do it.

图片 3

to check the version of your Ubuntu.  If you want to dual-boost OS from window10, like what I did, there is another note teach you how to do the dual-boost.  The following passages assume you have already had your Ubuntu 16.04 installed correctly.

Desktop clients

  • cmst — Qt GUI for ConnMan.

|| cmstAUR

  • connman-ncurses — Simple ncurses UI for ConnMan; not all of connman functionality is implemented, but usable (with X or from terminal without X), see the wiki.

|| connman-ncurses-gitAUR

  • connman-notify — Connman event notification client

|| connman-notifyAUR[[broken link](https://wiki.archlinux.org/index.php/ArchWiki:Requests#Broken_package_links): archived in [aur-mirror](https://github.com/felixonmars/aur3-mirror/tree/master/connman-notify)]

  • ConnMan-UI — GTK3 client applet.

|| connman-ui-gitAUR

  • connman_dmenu — Client/frontend for dmenu.

|| connman_dmenu-gitAUR

  • Econnman — Enlightenment desktop panel applet.

http://www.enlightenment.org || econnmanAUR

  • LXQt-Connman-Applet — LXQt desktop panel applet.

|| lxqt-connman-applet-gitAUR

  • qconnman-ui — Qt management interface used on O.S. Systems products

|| qconnman-ui-gitAUR[[broken link](https://wiki.archlinux.org/index.php/ArchWiki:Requests#Broken_package_links): archived in [aur-mirror](https://github.com/felixonmars/aur3-mirror/tree/master/qconnman-ui-git)]

  • connman-gtk — GTK client.

|| connman-gtkAUR

  • gnome-extension-connman — Gnome3 extension for connman; it contains only some of the functionality without installing connman-gtk.

||

3. How to use the bluez and hcitool


Usage

图片 4This article or section needs expansion.图片 5

Reason: Only Wired and Wi-Fi plugins are described. (Discuss in Talk:ConnMan#)

ConnMan has a standard command line client connmanctl. It can run in 2 modes:

  • In command mode commands are entered as arguments to connmanctl command, just like systemctl.
  • Interactive mode is started by typing connmanctl without arguments. Prompt will change to connmanctl> to indicate it is waiting for user commands, just like python interactive mode. The interactive mode supports tab completion, which makes finding the correct connection easy.

3.1 setup bluetooth service

Start the bluetooth service and enable automatic startup, assuming you're using systemd as the init daemon:

sudo systemctl start bluetooth.service
sudo systemctl enable bluetooth.service

Install pycharm

ref: https://itsfoss.com/install-pycharm-ubuntu/

Open a terminal and use the following commands:

sudo add-apt-repository ppa:mystic-mirage/pycharmsudo apt-get update

To install the community edition of PyCharm, use the command below. It will download data of around 120 MB.

sudo apt-get install pycharm-community

To install the professional edition of PyCharm, use the command below. It will download data of around 160 MB.

sudo apt-get install pycharm

Wired

ConnMan will automatically handle wired connections.

3.2 hcitool scan for bluetooth devices

Before start scanning make sure that your bluetooth device is turned on and not blocked, you can check that with the rfkill command:

sudo rfkill list

If the bluetooth device is blocked (soft or hard blocked), unblock it with the rfkill command again:

sudo rfkill unblock bluetooth

Bring up the bluetooth device with hciconfig command and start scanning, make sure the target device's bluetooth is on and It's discoverable:

sudo hciconfig hci0 up
hcitool scan

Wait few moment to complete the hcitool scan or hcitool lescan, the results will be something like bellow:

图片 6

图片 7

Here 00:1A:7D:DA:71:0A is the bluetooth MAC address and SHEN-PC is the name of the bluetooth device, i.e. an PC.

note: use hcitool lescan will forever scan ble devices, if use ctrl+c stop it, it will show error(ref to LINKS 4 to solve):

hcitool lescan
Set scan parameters failed: Input/output error

Remove PyCharm

To uninstall PyCharm installed by the above described method, use the command below to uninstall the community edition of PyCharm:

sudo apt-get remove pycharm-community

To remove the professional version, use the command below:

sudo apt-get remove pycharm

Afterward, use the command below to remove the PPA from the sources list:

sudo add-apt-repository --remove ppa:mystic-mirage/pycharm

That’s all you need to do. I hope this tutorial helped you toinstall PyCharm in Ubuntu Linux. Any questions or suggestions are always welcomed.


Install Python2.7

ref:

sudo add-apt-repository ppa:fkrull/deadsnakes

sudo apt-get update

sudo apt-get install python2.7


Wi-Fi

3.3 bluetooth service discovery

Now we have the bluetooth MAC address of the target device, use the sdptool command to know which services (like DUN, Handsfree audio) are available on that target device.

sdptool browse 28:ED:6A:A0:26:B7

You can also use the interactive bluetoothctl tool for this purpose.

If the target device is present, you can ping it with l2ping command, requires root privilege:

➜  bluez-5.50  sudo l2ping 94:87:E0:B3:AC:6F
Ping: 94:87:E0:B3:AC:6F from B8:27:EB:8E:CC:51 (data size 44) ...
44 bytes from 94:87:E0:B3:AC:6F id 0 time 53.94ms
44 bytes from 94:87:E0:B3:AC:6F id 1 time 77.12ms
44 bytes from 94:87:E0:B3:AC:6F id 2 time 38.63ms
44 bytes from 94:87:E0:B3:AC:6F id 3 time 46.13ms
44 bytes from 94:87:E0:B3:AC:6F id 4 time 59.96ms
5 sent, 5 received, 0% loss

So, bluetooth service discovery is useful to determine the type of the device, like if it's a bluetooth mp3 player or it's a keyboard.

图片 8

Install pip

Open Pycharm that you just installed.  Create a project.  Use Alt + F12or View -> Tool Windowto open the terminal.

Then follow the instructions in here: https://www.rosehosting.com/blog/how-to-install-pip-on-ubuntu-16-04/

1. Connect to SSH and Update your System Software

First of all, connect to your server via SSH and make sure that all your system software is up to date. Run the following command to update the package list and upgrade all your system software to the latest version available:

sudo apt-get update && sudo apt-get -y upgrade

2. Install Pip on Ubuntu 16.04

Once the upgrade is completed, you can move on and install Pip on your Ubuntu VPS. The installation of Pip is very simple. The only thing you need to do is to run the following command:

sudo apt-get install python-pip

3. Verify the Pip Installation on Ubuntu 16.04

The apt package manager will install Pip and all the dependencies required for the software to work optimally. Once the installation is completed you can verify that it was successful by using the following command:

pip -V

You should see something similar to the following:

# pip -V

pip 8.1.1 from /usr/lib/python2.7/dist-packages (python 2.7)

That means Pip has been successfully installed on your Ubuntu server and it is ready to use.


Enabling and disabling wifi

To check if wifi is enabled you can run connmanctl technologies and check for the line that says Powered: True/False. To power the wifi on you can run connmanctl enable wifi or if you need to disable it you can run connmanctl disable wifi. Other ways to enable wifi could include using the Fn keys on the laptop to turn it on or running ip link set <interface> up.

LINKS

  • 1.Bluetooth on Modern Linux by Szymon Janc
  • 2.dbus-python tutorial
  • 3.Linux bluetooth setup with bluez and hcitool
  • 4.hcitool lescan shows I/O error

图片 9

@beautifulzzzz
智能硬件、物联网,热爱技术,关注产品
博客:http://blog.beautifulzzzz.com
园友交流群:414948975

Install numpy, tdpm, pillow, scrip etc...

Go to File -> Settings

In Project: -> Project interpreter, make sure you have a python2.7 selected.  Click the green plus sign at the right.  Then search whatever the package you need to install and click Install Package button at the bottom of the new page you just opened to mount it.

Caution: To run tensorflow using GPU, better to do not install it from pycharm.  See the UPDATE below how to install it.


Connecting to an open access point

The commands in this section show how to run connmanctl in command mode.

To scan the network connmanctl accepts simple names called technologies. To scan for nearby Wi-Fi networks:

$ connmanctl scan wifi

To list the available networks found after a scan run (example output):

$ connmanctl services

*AO MyNetwork               wifi_dc85de828967_68756773616d_managed_psk
    OtherNET                wifi_dc85de828967_38303944616e69656c73_managed_psk 
    AnotherOne              wifi_dc85de828967_3257495245363836_managed_wep
    FourthNetwork           wifi_dc85de828967_4d7572706879_managed_wep
    AnOpenNetwork           wifi_dc85de828967_4d6568657272696e_managed_none

To connect to an open network, use the second field beginning with wifi_:

$ connmanctl connect wifi_dc85de828967_4d6568657272696e_managed_none

Tip: Network names can be tab-completed.

You should now be connected to the network. Check using ip addr or connmanctl state.

Installing CUDA Toolkit 8.0 on Ubuntu 16.04

ref: http://www.pradeepadiga.me/blog/2017/03/22/installing-cuda-toolkit-8-0-on-ubuntu-16-04/

GCC

One of them is to ensure where GCC is installed or not. We can confirm it by executing the following command.

gcc --version

Since I am using Ubuntu, GCC comes pre-installed and here is the output that I got.

图片 10

build essentials

It is important have thebuild-essential package installed. This is usually pre-installed on Ubuntu, however if it is not you can install it by executing the following command.

sudo apt-get install build-essential

On my laptop it was already installed hence I got the following output.

图片 11

Download CUDA package from NVIDIA website

Navigate to https://developer.nvidia.com/cuda-downloads and download the appropriate package.

图片 12

Once the page is successfully downloaded, we need to install the package. First navigate to the folder where the package is located. In my case it is under ~/Downloads/CUDA$ folder. Then issue the following command which installs the package.

sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb

图片 13

Then update the package list from the repositories using the below command.

sudo apt-get update

Then install CUDA by executing the following command.

sudo apt-get install cuda

图片 14

After a couple of minutes the installation would succeed and you should a screen similar to the following.

图片 15

One of the important post installation steps is to update the PATH variable to include the CUDA binaries folder. To update it, we need to edit the file /etc/environment. I use the nano text editor in this post, so the command would be

sudo nano /etc/environment

Once nano is open edit the PATH variable to include /usr/local/cuda-8.0/bin folder. After editing the file screen would look like this.

图片 16

After editing this line press Ctrl + X to exit the editor and press Y when prompted whether you want to save it.

This method of editing the PATH variable usually requires a reboot to take effect. However executing the below command would update the PATH variable immediately.

source /etc/environment

Now we are ready to validate the CUDA installation. Just execute the following command in the terminal.

nvcc --version

If the installation was successful, we should see the CUDA compiler version as seen in this screenshot.

图片 17

Alternatively you can also execute the following command in the terminal. This gives more detailed information about the drivers.

nvidia-smi

图片 18

We are now ready to enjoy the goodness of CUDA and can continue with the installation of TensorFlow. Stay tuned for the installation instructions of TensorFlow.


Connecting to a protected access point

For protected access points you will need to provide some information to the ConnMan daemon, at the very least a password or a passphrase.

The commands in this section show how to run connmanctl in interactive mode, it is required for running the agent command. To start interactive mode simply type:

$ connmanctl

You then proceed almost as above, first scan for any Wi-Fi technologies:

connmanctl> scan wifi

To list services:

connmanctl> services

Now you need to register the agent to handle user requests. The command is:

connmanctl> agent on

You now need to connect to one of the protected services. To do this easily, just use tab completion for the wifi_ service. If you were connecting to OtherNET in the example above you would type:

connmanctl> connect wifi_dc85de828967_38303944616e69656c73_managed_psk

The agent will then ask you to provide any information the daemon needs to complete the connection. The information requested will vary depending on the type of network you are connecting to. The agent will also print additional data about the information it needs as shown in the example below.

Agent RequestInput wifi_dc85de828967_38303944616e69656c73_managed_psk
  Passphrase = [ Type=psk, Requirement=mandatory ]
  Passphrase?  

Provide the information requested, in this example the passphrase, and then type:

connmanctl> quit

If the information you provided is correct you should now be connected to the protected access point.

Install cudnn v5.1 (you may require for cudnn v6.0 too, see the update below)

ref:

Download cudnn @https://developer.nvidia.com/rdp/cudnn-download

My os is Ubuntu 16.04.  amd64.  The debian archives for Ubuntu 16.04 Power8 doesn't apply to my os.  So I have to download the tar file for cuDNN v5.1 Library for Linux.  (This is under the testing.  If it doesn't work.  I have to change os to Ubuntu 14.04 cuz there are debian archives for amd64 version)

Next you need to uncompress and copy cuDNN to the toolkit directory. The toolkit default install location is /usr/local/cuda

tar xvzf cudnn-8.0-linux-x64-v5.1-ga.tgz

sudo cp -P cuda/include/cudnn.h /usr/local/cuda/include

sudo cp -P cuda/lib64/libcudnnInstalling Pycharm, Python Tensorflow, Cuda and cudnn in Ubuntu 16.04。 /usr/local/cuda/lib64*

*sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn**

Now you need to update your bash file

nano ~/.bashrc

With the text editor open, scroll to the bottom and put in these lines:

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64"

export CUDA_HOME=/usr/local/cuda

Save and close it.


Settings

Settings and profiles are automatically created for networks the user connects to often. They contain fields for the passphrase, essid and other information. Profile settings are stored in directories under /var/lib/connman/ by their service name. To view all network profiles run this command from root shell:

# cat /var/lib/connman/*/settings

Note: VPN settings can be found in /var/lib/connman-vpn/.

UPDATE:

Technologies

Various hardware interfaces are referred to as Technologies by ConnMan.

To list available technologies run:

$ connmanctl technologies

To get just the types by their name one can use this one liner:

$ connmanctl technologies | awk '/Type/ { print $NF }'

Note: The field Type = tech_name provides the technology type used with connmanctl commands

To interact with them one must refer to the technology by type. Technologies can be toggled on/off with:

$ connmanctl enable technology_type

and:

$ connmanctl disable technology_type

For example to toggle off wifi:

$ connmanctl disable wifi

Warning: connman grabs rfkill events. It is most likely impossible to use rfkill or bluetoothctl to (un)block devices, yet hardware keys may still work.[1] Always use connmanctl enable|disable

What to check?

To run gpu using tensorflow under Pycharm IDE, here is a checklist:

1, test whether CUDA and cudnn run correctly.

2. test whether CUDA could pick up your GPU device.

3, make sure whether tensorflow from Pycharm can see Cuda and cudnn

Solutions:

TO "1", install cuda and cudnn in the correct version as above says. Use nvcc -V and nvidia-smi command to examine the driver and installation.

Run sample code to test whether cuda works correctly

ref:http://xcat-docs.readthedocs.io/en/stable/advanced/gpu/nvidia/verify_cuda_install.html

-> Go to the dir that installed cuda (mine is/usr/local/cuda/samples)

-> runmake, it might take few minutes to make the files.

-> rundeviceQueryandbandwidthTest. They are at:

./bin/ppc64le/linux/release/deviceQuery

./bin/ppc64le/linux/release/bandwidthTest

if both passes, then cuda should work.

TO "2", run the testing code provided from the official site of Tensorflow:

ref:https://www.tensorflow.org/tutorials/using_gpu

write down the code below and run through terminal:

import tensorflow as tf

# Creates a graph.

a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')

b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')

c = tf.matmul(a, b)

# Creates a session with log_device_placement set to True.

sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

# Runs the op.

print(sess.run(c))

Save as a .py file and run python ./${file_full_path}

You should see the following output:

Device mapping:

/job:localhost/replica:0/task:0/device:GPU:0 -> device: 0, name: Tesla K40c, pci bus

id: 0000:05:00.0

b: /job:localhost/replica:0/task:0/device:GPU:0

a: /job:localhost/replica:0/task:0/device:GPU:0

MatMul: /job:localhost/replica:0/task:0/device:GPU:0

[[ 22.  28.]

[ 49.  64.]]

If it shows device mapping: no known devices, then unintall your tensorflow and install tensorflow-gpu.  See this:

ref:https://stackoverflow.com/questions/42326748/tensorflow-on-gpu-no-known-devices-despite-cudas-devicequery-returning-a-pas

Package tensorflow is likely a CPU version.  To use gpu to run CUDA, you need to install tensorflow-gpu.  However, since installing tensorflow-gpu will reinstall tensorflow automatically, it'd be wise to manually uninstall all the tensorflow-related package, then install tensorflow-gpu.

In my case:

pip uninstall tensorflow

is not enough. Because when reinstall with:

pip install tensorflow-gpu

It's still reinstall tensorflow with cpu not gpu. So, before install tensorfow-pgu, I tried to remove all related tensor folders in site-packages uninstall protobuf, and it works!

For conclusion:

pip uninstall tensorflow

Remove all tensor folders in ~/Python35/Lib/site-packages

pip uninstall protobuf

pip install tensorflow-gpu

If the commandpip uninstall tensorflow results in some kind of error and failed in uninstall, then we can remove the package from pycharm.  Go File-> settings-> project interpreter->minus sign to uninstall.  Also check the fold from terminal whether they are still there. /usr/local/lib/python2.7/dist-packages/ and /usr/local/lib/python2.7/site-packages/.  List of folders need to remove:

*tensorflow,tensorboard,anything starts from **tensor, and protobuf*.

Open NVIDIA X Server Settings.  At GPU-0 (your gpu name) tab, monitor your GPU Utilization.  If you see it rapidly goes up in a very short period of time, it means your tensorflow runs on gpu correctly.

If you see error message occurs while you import tensorflow like:

ImportError: libcudnn.so.5: cannot open shared object file: No such file or directory

or

ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory

it means it couldn't find the corresponding version of cudnn library.  It might trying to load cudnn v6.  If so,Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0, untar and place the files to the corresponding right places as above (since cudnn v5.1 and v6.0 can be exist at the same time, you don't have to delete the old copied files from v5.1) and run the tensorflow code again.  It should be fixed.

TO "3",  it is possible that everything works when you run through terminal, but it suddenly breaks when you run through pycharm although you have already added LD_LIBRARY_PATH and CUDA_HOME to ~/bashrc file.  It is because pycharm has its own environment variable Running in pycharm, we need to set CUDA_HOME and LD_LIBRARY_PATH again probably for each individual project.

To do that, go to the Run menu->edit configurations then choose your project. Then you click on the Environment Variables and add an entry for CUDA_HOME and LD_LIBRARY_PATH.

ref:https://stackoverflow.com/questions/43691706/pycharm-tensorflow-importerror-but-works-fine-with-terminal

Up to now, your configuration of cuda and tensorflow should be done.  Make sure you have enabled to use gpu in your code, then everything should work.

本文由永利集团登录网址发布于网络安全,转载请注明出处:Installing Pycharm, Python Tensorflow, Cuda and cudnn in Ubu

关键词: