Python

(environments, libraries, and getting started)

Quality coding #2

Dan Fitch

2020-09-22

Reminder: Python 3

Python 2 is old now, y'all!

Environments

What are some goals of a programming environment?

Maintainable, reusable code
Packagable so you can distribute it to others
Dependencies are clear and easy to install

But everyone has different assumptions...

There are... a lot of ways to do this.

Mushy distinctions:

python version and binaries
python packages
project environments

Some possible ways

to set up your environment and packages

Python binaries

(installed by default on Macs and Linux, but maybe you need a newer version than that!)

pip install

(great for grabbing libraries on your own machine)

pip install --user

(great for your own account on someone else's machine)

venv

(in base python3 library but still too limited to be useful)

virtualenv

(the old standard for package isolation)

conda (anaconda, miniconda)

(combination of python binary isolation and package isolation)

what we usually recommend for studies, see Python in the wiki)

pyenv

(python version isolation with less overhead than conda)

pyenv-virtualenv

(link between pyenv and virtualenv, also deals with conda envs)

poetry

(a new opinionated all-in-one standard)

pipenv

(another popular attempt to join all these concerns in one place)

R and Python?

reticulate

use Python inside R

(integrates with virtualenv and conda environments)

But what about the other way???

rpy2

Do your glue and frame in python, and your stats in R

(disclaimer: I have not used this AT ALL but it seems pretty easy and works inside notebooks)

Prepare your conda!

Create an environment

conda create --name example

conda env list

conda activate example

Install jupyter notebook

conda install jupyter

(how is this different from pip install?)

jupyter notebook

Ctrl-C!

Security and notebooks

Keep your key safe!

Those ports are usually firewalled by default, but anyone on the same machine could pretend to be you if you publish or screenshare your key.

Shared notebooks? Advantages and disadvantages...

Actual multi-tenant: Jupyter Hub

Using Jupyter notebook

Running code

Run with Shift-Enter

Run all with the Run button

Useful shortcuts

Ctrl-Shift-F: Searchable command list

Code

What can you do in your notebook?

Learn python!

Define and use functions:

Load data and plot it:

Higher resolution and size of plot defaults:

Experiment with a library that's new to you:

Exercise

File looper

Some ideas, of increasing difficulty

  • Print the filenames
  • Copy the files to a different folder
  • Copy the files to a different folder, but check if they already exist and rename them if needed
  • Make a plot of file sizes
  • Extract EXIF header information from your personal photographs
  • Make a recursive plot of file sizes that drills into directories

What next?

Export an HTML or PDF file

Export to reveal.js slides

Turn it into a reusable .py script

If we have time...

Installing miniconda on BI servers

Using your own personal conda vs. a study-wide conda and set_study?

Personal miniconda on BI servers

Start a terminal to the login servers
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh ... hit ENTER, space to view the license, type "yes"
For location, put
~/apps/conda3.8
Hit enter at the final prompt about activating
Once installed, run
conda update -n base -c defaults conda

SSH tunnel

What server am I on?

hostname

On a Mac or Linux machine:

ssh -L 8889:localhost:8889 USERNAME@SERVER.keck.waisman.wisc.edu

(replace USERNAME and SERVER)

On a Windows machine with putty:

TODO: google "SSH tunnel putty"

Now run jupyter notebook and open the link

THE END

(discussion)

wheeeeee