Python

(environments, libraries, and getting started)

Quality coding #2

Dan Fitch

2020-09-22

Reminder: Python 3

Python 2 is old now, y'all!

Environments

What are some goals of a programming environment?

Maintainable, reusable code

Packagable so you can distribute it to others

Dependencies are clear and easy to install

But everyone has different assumptions...

There are... a lot of ways to do this.

Mushy distinctions:

python version and binaries

python packages

project environments

Some possible ways

to set up your environment and packages

Python binaries

(installed by default on Macs and Linux, but maybe you need a newer version than that!)

pip install

(great for grabbing libraries on your own machine)

pip install --user

(great for your own account on someone else's machine)

venv

(in base python3 library but still too limited to be useful)

virtualenv

(the old standard for package isolation)

conda (anaconda, miniconda)

(combination of python binary isolation and package isolation)

what we usually recommend for studies, see Python in the wiki

pyenv

(python version isolation with less overhead than conda)

pyenv-virtualenv

(link between pyenv and virtualenv, also deals with conda envs)

poetry

(a new opinionated all-in-one standard)

pipenv

(another popular attempt to join all these concerns in one place)

R and Python?

reticulate

use Python inside R

(integrates with virtualenv and conda environments)

But what about the other way???

rpy2

Do your glue and frame in python, and your stats in R

(disclaimer: I have not used this AT ALL but it seems pretty easy and works inside notebooks)

Prepare your conda!

Create an environment

`conda create --name example`

`conda env list`

`conda activate example`

Install jupyter notebook

`conda install jupyter`

(how is this different from pip install?)

`jupyter notebook`

`Ctrl-C!`

Security and notebooks

Keep your key safe!

Those ports are usually firewalled by default, but anyone on the same machine could pretend to be you if you publish or screenshare your key.

Shared notebooks? Advantages and disadvantages...

Actual multi-tenant: Jupyter Hub

Using a sample Jupyter notebook

Running code

Run with Shift-Enter

Run all with the Run button

Restarting the kernel - what?

Useful shortcuts

Ctrl-Shift-F: Searchable command list

Using the notebook

What can you do in your notebook?

Write notes in Markdown!

Learn python!

Define and use functions:

Load data and plot it:

Higher resolution and size of plot defaults:

Experiment with a library that's new to you:

What next?

Export an HTML or PDF file

Export to reveal.js slides

Turn it into a reusable `.py` script

If we have time...

Exercise: file looper

Print the filenames
Copy the files to a different folder
Copy the files to a different folder, but check if they already exist and rename them if needed
Make a plot of file sizes
Extract EXIF header information from your personal photographs
Make a recursive plot of file sizes that drills into directories

Installing miniconda on BI servers

Using your own personal `conda` vs. a study-wide `conda` and `set_study`?

Personal miniconda on BI servers

Start a terminal to the login servers

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

bash Miniconda3-latest-Linux-x86_64.sh ... hit ENTER, space to view the license, type "yes"

For location, put
~/apps/conda3.8

Hit enter at the final prompt about activating

Once installed, run
conda update -n base -c defaults conda

SSH tunnel

What server am I on?

`hostname`

On a Mac or Linux machine:

ssh -L 8889:localhost:8889 USERNAME@SERVER.keck.waisman.wisc.edu

(replace USERNAME and SERVER)

On a Windows machine with putty:

TODO: google "SSH tunnel putty"

Now run jupyter notebook and open the link

THE END

(discussion)

wheeeeee