Jupman

A template for online books made with Jupyter notebooks and NBSphinx doc generator.

DOWNLOAD: PDF EPUB HTML Github

Features

inherits generation of static websites from NBSphinx, with search, PDF, EPUB
builds with ReadTheDocs, Github Actions, or local Docker emulating ReadTheDocs
exercises generation from solution templates (both .ipynb and .py)
zips chapters
code sharing among chapters (so students don’t need to install dependencies)
Python Tutor integration (offline, no required dependencies)
supports hundreds pages and deep nesting
decent PDF layout
basic exam management
optional softpython theme
documentation and tests
Apache License v2.0, open source code on Github

Currently lacking

Requirements

Python 3.7+
based upon sphinx-rtd-theme (adds only minimal improvements for better browsing)

Used by

SoftPython book: english | italian (both 1000+ pdf pages from jupyter notebooks)
Scientific Programming Lab @ University of Trento, Data Science Master (english, many python exercises and exams in mixed jupyter + python format)

Contents

Quickstart
Editing worksheets
Customize
Exams management
Chapter examples
1. Python example
2. Jupyter example
3. Mixed jupyter and python example
4. Challenges: example worksheet solution
Templates
1. Past exams
2. Changelog
Tests
1. Rendering tests
2. Python Tutor tests

Revisions

19 September 2022: Released v3.5.6
4 June 2022: Released v3.5
30 April 2022: Released v3.4
22 February 2022: Released v3.3
16 October 2020: Released v3.2
16 January 2020: Released v3.1
29 December 2019: Released v3.0
24 September 2018: Released v2.0
3 August 2018: Released v0.8
Change log

Credits

This site was made with Jupyter using NBSphinx extension and Jupman template.

Quickstart

Jupman uses NbSphinx and either ReadTheDocs or Github Actions

Installation

(Instructions are for Ubuntu, on Windows may differ)

First, on Github, fork as a template jupman project to create yours, for example my-project.

Then, on your computer, clone the my-project from Github

You can choose to build either on:

ReadTheDocs
Github Actions
locally with plain Sphinx
locally with RTD Docker

(Note Jupman itself is building on both ReadTheDocs and Github Actions only for testing purposes, one is enough)

Building with ReadTheDocs:

IMPORTANT: choose a name which is NOT already on ReadTheDocs

Create a ReadTheDocs account using the same name as in Github so the address in readthedocs will be something like my-project.readthedocs.org.

Use ReadTheDocs panels to link the project to your Github repository.
In Admin-> Advanced settings panel, set:
- Python interpreter to CPython 3.x
- Requirements to requirements-build.txt

Building with Github Actions:

Configure .github/workflows/main.yml on your computer to your needs - you will need to:

at the beginning in the build_docs_job section there is an if which makes the workflow only work in DavidLeoni/jupman repository, change it with your project repo and comment the following needs: run_tests line
set RTD_PRJ_NAME
If you want to publish to Github Pages: everything is set, just create an empty branch gh-pages in an new HTML_FOLDER before committing - from some other folder in your file system:

git clone  YOUR_REPO_ADDRESS  HTML_FOLDER
cd HTML_FOLDER
git checkout --orphan gh-pages
git rm -rf .
touch bla
git add .
git commit -m "init"
git push origin gh-pages

Local build with Sphinx

Assuming you are on Linux/Mac:

Install Python 3.7+
Install Jupyter
Install required modules:

3.a) In a virtual environment (recommended) - from the root of the project, run:

./create-env.sh

This will automatically install required modules in _private/jupman_env using python3 system binary.

If you want to usa a particular python binary (note it must already be on your system), run i.e.:

./create-env.sh python3.7

Afterwards, to activate the environment run:

In Windows:

_private\jupman_env\bin\activate

in Linux/Mac:

source activate

to deactivate (from anywhere):

deactivate

3.b) without a virtual environment (not recommended): From the root of the project, run:

python3 -m pip install --user -r requirements-build.txt

Warning: to have reproducible builds requirements-build.txt pinpoints a lot of dependencies , installing without virtual environment may mess up other python programs!

Optional - Running tests

To check everything is working, you may want to run the tests.

Install test dependencies:

python3 -m pip install --user -r _test/requirements-test.txt

Run the tests:

python3 -m pytest _test/*_test.py

Optional - Install Jupyter contrib extensions

For a better editing experience like having Table of contents and other things, do the following:

install the Jupyter contrib extensions package:

If you have Anaconda:

conda install -c conda-forge jupyter_contrib_nbextensions

If you don’t have Anaconda:

python3 -m pip install --user jupyter_contrib_nbextensions

Install in Jupyter:

jupyter contrib nbextension install --user

Enable extension:

For being able to view table of contents while editing notebooks, install toc2 extension:

jupyter nbextension enable toc2/main

For tocs to appear when in a document you will need to press a list button at the right-end of the toolbar.

(since Jupman 0.8 custom injected tocs are disabled by default)

For a nice GUI to install extensions, install the Jupyter Nbextensions configurator:

If you have Anaconda:

From Anaconda Prompt:

conda install -c conda-forge jupyter_nbextensions_configurator

If you don’t have Anaconda:

python3 -m pip install --user jupyter_nbextensions_configurator

After installing, enable it:

jupyter nbextensions_configurator enable --user

and then start Jupyter, in file browser look for a Nbextensions tab

Configure

Edit conf.py as needed, which is the configuration file for Sphinx. In particular, you MUST edit the sections marked with TODO
Try launching build:
```
python3 build.py
```
For more info, see related section
If everything works fine on your computer, push changes back to Github
Go back to ReadTheDocs and try to run a build. Hopefully your project will become available on something like my-project.readthedocs.org
If you want to grade exams, see Exams management page.

You should now be ready to create your notebooks by launching from the project root:

jupyter notebook

If you wish your notebooks to appear in the generated manual, you have to add them in the toc.rst file.

NOTE: the page toc-page.rst, which is set to be the master_doc of Sphinx, will just load the actual Table of Contents which is in toc.rst. It looks a bit convoluted because when it comes to indexes Sphinx is not much reliable, see this issue . We strongly advise not to change these settings !
edit the home, which is in the index.ipynb file

Building the manual

For a quick build that only produces html:

python3 build.py -q

Site will be created in _build/ folder.

For help:

python3 build.py -h

To build everything (html + pdf + epub), go to the console and from the root directory run:

python3 build.py

NOTE: to generate PDFs you will need to install Latex environment

Publish

Just push to master and Github Actions / ReadTheDocs will do the rest, for example, for jupman is available at addresses:

IMPORTANT: ReadTheDocs WILL *NOT* execute Jupyter notebooks because of this bug

Further steps

See Editing worksheets, Customize and if needed Exams management pages

Editing worksheets

Here we give an overview of how to edit worksheets. More info can be found in Tests notebook

Common files

There are a bunch of files common to all worksheets and possibly website

You do not need to change them (except maybe my_lib.py)

File	Description	Jupyter	Website	Student zips
jupman_tools.py	back end stuff	X
jupman.py	utilities for worksheets	X		X
my_lib.py	custom utilities example	X		X
_static/js/jupman.js	Javascript code	X	X	X
_static/css/jupman.css	CSS	X	X	X
_static/css/jupman-web.css	CSS		X

Running Jupyter

First of all, run Jupyter from the root directory:

jupyter notebook

Source code for chapters

Put chapters one per folder, in the root. Any folder which doesn’t start with underscore _ or exam/ will be considered a chapter.

During build, each chapter gets automatically zipped and zip goes to _static/generated. So for example, python-example/ produces a zip called _static/generated/python-example.zip, which will have these contents:

python-example
    _static
       js
          jupman.js
          toc.js
       css
          jupman.css
       img
          cc-by.png
    python-example.ipynb
    lab.py
    lab_test.py
    lab_sol.py
    jupman.py
    my_lib.py

The zip folder structure will be a merge of chapter files and files shared by all chapters which are specified in exercises_common_files variable in conf.py. Since the root in the zip becomes the chapter itself, jupman will process .py and .ipynb files for fixing eventual relative imports. Markdown, HTML and CSS links in ipynb will also be adjusted.

Exercise files can be automatically generated from solutions, as we will see next.

Exercise types

There can be three kinds of exercises: exercises in Python files, exercises in Jupyter files and mixed jupyter and Python exercises.

You can automatically generate an exercise from a solution file by stripping text marked with special tags. You can inspect generated files in _build/jupman/ directory

On the website, students will be able to see solutions by clicking on appropriate buttons.

In the zips to download, two versions of files will provided, one without solution and one with solutions (in exam modality of course no solution will be shipped)

Exercises in Python files

See python-example/python-example.ipynb

In this type of exercises, typically you have a Jupyter file (like python-example.ipynb) that describes the exercise and then the actual exercises are in Python files.

If there is a solution file FILE_sol.py ending in _sol.py but no corresponding exercise file FILE.py without the _sol:

then Jupman will try to generate FILE.py one from FILE_sol.py. To do so, it will look for tags to strip inside the solution file.

If there is already an exercise file like this:

python_intro.py
python_intro_sol.py

Jupman will just copy the existing file.

Exercises in Jupyter files

See example: jupyter-example/jupyter-example-sol.ipynb

This type of exercises stay in a Jupyter notebook itself.

If there is a notebook ending in -sol.ipynb, the following applies (WARNING: for ipynb files we use dash -, not the underscore _):

the notebook must contain tags to strip
exercises derived will have ‘EXERCISES’ appended to the title (the word can be customized in conf.py - you might need to translate it)

Mixed exercises in Jupyter and Python files

See jup-and-py-example/jup-and-py-example-sol.ipynb

Challenges

This is an experimental feature, current implementation is subject to change.

Challenges are solutions which remain unpublished and from which exercises are generated in the same original older where the solution resides (not only in the zip!). Challenge files can be both Jupyter notebooks or Python files, ending in -chal-sol.ipynb or _chal_sol.py.

The idea is that challenges solutions are gitignored, and exercises are manually generated by calling jupman.generate_exercise() inside a Jupyter notebook like this:

#jupman-purge
import sys; sys.path.append('../'); from conf import jm;
jm.generate_exercise('great_chal_sol.py')
#/jupman-purge

It is a bit laborious but the idea is that typically you will also want to run and see tests results in Jupyter notebook so you can do it in the same final cell, which you will also probably want to set in cell metadata "nbsphinx":"hidden"

the solution notebook must contain tags to strip and have SOLUTIONS at the end of the title (the word can be customized in conf.py - you might need to translate it)

Solution tags

Presence of these tags marks a cell as a solution.

Start tags begin with a # while end tags begin with a #\

jupman-raise

Replaces code inside with an Exception (text is customizable in conf.py). Be careful to position the comment exactly with the indentation yuoi want the raise to appear. For example:

def add(x,y):
    #jupman-raise
    return x + y
    #/jupman-raise

becomes

def add(x,y):
    raise Exception('TODO IMPLEMENT ME !')

jupman-strip

Just strips code inside exercises

def f(x):
    print(x)

#jupman-strip
def help_func(x,y):
    return x - y
#/jupman-strip

def g(y):
    return y

becomes

def f(x):
    print(x)

def g(y):
    return y

write here

This special tag for python code erases whatever is found afterwards the # write here line

you can put how many spaces you want in the comment
phrase can be customized in conf.py

w = 5

#  write  here  fast please

x = 5 + w
y = 2 + x

becomes

w = 5

#  write  here  fast please

SOLUTION

In a code cell, if you put # SOLUTION at the beginning the whole cell cell content gets deleted (# SOLUTION string included).

Word can be customized in conf.py

# SOLUTION

def f():
    print('hello')

becomes nothing:

[ ]:

QUESTION - ANSWER

In a markdown cell, everything in a cell with **ANSWER**: inside will be stripped.

Markdown can be customized in conf.py

QUESTION: Describe why iPhone n+1 is better than iPhone n

ANSWER: it costs more

Becomes:

QUESTION: Describe why iPhone n+1 is better than iPhone n

[ ]:

Directive tags

Some tags change the preprocessor behaviour. They are applied before solution tags.

jupman-purge

Eliminate content both from exercises AND solutions. Can be helpful when you have code which creates expected output, like images or python data - the idea is to completely remove code so so students don’t accidentally copy-paste it or uncomment it.

jupman-purge-input: purges only cell source
jupman-purge-output: purges only cell output
jupman-purge-io : purges both input and output

jupman-purge purges only a span:

x=5
#jupman-purge
plt.savefig('expected_image.png')
jupman.save_py('expected_output_db.py', ['big', 'data', 'structure']*1000)
#/jupman-purge
x=6

becomes

x=5
x=6

jupman-preprocess

By default only notebooks solutions (ending in -sol.ipynb) are preprocessed before html conversion begins. If you want to force preprocessing on a particular non-solution notebook, add this in the first cell:

#jupman-preprocess

Hiding cells

A way to hide cells (like for example the import jupman code) is by clicking View->Cell toolbar -> Edit metadata and adding "nbsphinx": "hidden" to the JSON (see also original NBSphinx docs and Togglable cells in Jupman tests ).

NOTE 1: As of NBSphinx 2.17, it is not possible to hide only cell text but not the output.

Info boxes

Supported boxes are inherited from NBSphinx with div classes "alert alert-info", "alert alert-warning"

See Rendering tests for examples.

Plus we add jupman-alert-principle: some alerts to be often reminded can be preceded with an empty div having class jupman-alert-principle followed by a regular alert box, so they will display as you want on the website and as fallback boxes in the pdf (did this way as we can’t add classes nor other attributes, tried also data-jupman html attributes with no success)

NOTE: default colors are indicative and minimal on purpose, for a better view see softpython themed version

Recommended approach: The typical principle alert should be brief and may have a link to more substantial text, with a short line under it. If you need more explicative text, put it outside:

<div class="jupman-alert-principle"></div>
<div class="alert alert-info">

[IV PRINCIPLE](https://jupman.softpython.org/principles.html#V-PRINCIPLE): **You shall write tests!**

Who does **not** writes tests, falls into _Debugging Hell_!
</div>

IV PRINCIPLE: You shall write tests!

Who does not writes tests, falls into Debugging Hell!

Utilities

NOTE: not mandatory, it’s mostly intended to tweak notebooks downloaded locally. Should be avoided in notebooks meant for students, as it’s more likely it will mess their configurations - also, they might copy the notebooks without knowing they contain the custom js and use weird extensions which could generate conflicts (such as double toc)

For notebooks in the root folder:

import jupman

Worksheets in in subfolders can use sys.path to locate the module

import sys
sys.path.append('../')
import jupman

Some reason for this ugliness is reported in this issue.

Launch unit tests

Inside worksheets you can run unittest tests.

To run all the tests of a test class, write like this

jupman.run(NameOfTheTestClass)

To run a single method, write like this:

jupman.run(NameOfTheTestClass.nameOfTheMethod)

Python Tutor

Among the various ways to embed Python Tutor, we decided to implement a special jupman.pytut() method. First you need to import the jupman module:

[1]:

import sys
sys.path.append('../')
import jupman

Then you can put a call to jupman.pytut() at the end of a cell, and the cell code will magically appear in python tutor in the output (except the call to pytut() of course). To see Python Tutor you don’t need to be online

[2]:

x = [5,8,4]
y= {3:9}
z = [x]

jupman.pytut()

[2]:

Python Tutor visualization

Beware of variables which were initialized in previous cells, they won’t be available in Python Tutor and you will get an error:

[3]:

w = 8

[4]:

x =  w + 5
jupman.pytut()

Traceback (most recent call last):
  File "../jupman.py", line 2453, in _runscript
    self.run(script_str, user_globals, user_globals)
  File "/usr/lib/python3.7/bdb.py", line 578, in run
    exec(cmd, globals, locals)
  File "<string>", line 2, in <module>
NameError: name 'w' is not defined

[4]:

Python Tutor visualization

Pandas

Correctly rendering pandas in PDFs is not so easy (see issue), so far we created this little function which sometimes is handy:

[5]:

import pandas as pd


lista = [['Rosanna', 'Gippalanda', 26, 100, 500, 300, 600, 600, 100, 300, 600, 300, 200, 400, 200, 300, 400, 500],
         ['Matilda', 'Zampola',    10, 500, 200, 300, 500, 400, 300, 200, 500, 300, 200, 400, 200, 300, 400, 500],
         ['Mario', 'Cipolli',      25, 300, 500, 100, 500, 300, 500, 100, 500, 300, 200, 400, 200, 300, 400, 500],
         ['Ugo', 'Sgarapirri',     30, 100, 400, 200, 500, 300, 200, 600, 300, 300, 200, 400, 200, 300, 400, 500]
        ]

df = pd.DataFrame(lista, columns =['Name', 'Surname', 'Age', *['Par'+str(i) for i in range(1,16)]])
df # web

[5]:

	Name	Surname	Age	Par1	Par2	Par3	Par4	Par5	Par6	Par7	Par8	Par9	Par10	Par11	Par12	Par13	Par14	Par15
0	Rosanna	Gippalanda	26	100	500	300	600	600	100	300	600	300	200	400	200	300	400	500
1	Matilda	Zampola	10	500	200	300	500	400	300	200	500	300	200	400	200	300	400	500
2	Mario	Cipolli	25	300	500	100	500	300	500	100	500	300	200	400	200	300	400	500
3	Ugo	Sgarapirri	30	100	400	200	500	300	200	600	300	300	200	400	200	300	400	500

[6]:

import jupman
jupman.draw_df(df)  # image for pdf

Showing function help

Python help is already quite good, but adds two useless extra lines and only works as a print, so we defined jupman.get_doc:

[7]:

print(jupman.get_doc(jupman.get_doc))

def get_doc(fun):
    """ Returns the help of a function formatted in a faithful manner

        @since 3.3
    """

Custom js and css

If you need custom js and/or css in a notebook, you can inject it by running

jupman.init()

in the first cell, it will inject jupman.js and jupman.css

Show table of contents

Since 0.8, custom toc is disabled, try instead installing toc2 extension. If you still want the jupman toc (not recommended), execute

jupman.init(toc=True)

it will create the sidebar even when editing in Jupyter. To refresh the sidebar, just rerun the cell.

Note: hiding the jupman.init code cell will prevent the build system to embed the Javascript file jupman.js inside the page in the HTML website: this is still fine as it is fetched separately by settings in conf.py.

Customize

Change website theme

If you want to change site colors and other changes, copy/edit _static/css/jupman-web.css and set it in conf html_css_files:

html_css_files = [
    'css/jupman.css',      # shared among jupyter and website
    'css/jupman-web.css',  # only on website
    #'css/softpython-theme.css',  # uncomment to activate
    #'css/scifi-theme.css',
]

Fonts

Fonts are a bit of a complex topic

TODO this part is just a collection of personal notes

Tools:

Comprehensive article: https://www.useragentman.com/blog/2011/02/20/converting-font-face-fonts-quickly-in-any-os/ and https://www.useragentman.com/blog/the-css3-font-converter/

https://github.com/zoltan-dulac/css3FontConverter

woff2

https://github.com/google/woff2

sfnt2woff

sudo apt-get install  libbrotli-dev
sfnt2woff  SomeFont.otf

mkeot

sudo apt-get install eot-utils
mkeot SomeFont.otf > SomeFont.eot

or https://github.com/wget/ttf2eot

FontForge (GUI and scriptable)

sudo apt-get install fontforge

font sizes

https://www.24a11y.com/2019/pixels-vs-relative-units-in-css-why-its-still-a-big-deal/

https://chiamakaikeanyi.dev/sizing-in-css-px-vs-em-vs-rem/

Warning about old versions

ReadTheDocs has a mechanism to warn the user if he’s looking at an old version of the site, but we found it doesn’t work much for course-based documentation. So for versioning we think it’s better to adopt a mixed git branch / tags devlopment model, and we added a template warning to show in old branches. To enable it in an old branch, just edit _templates/breadcrumbs.html as needed.

Fix nbsphinx to create rst files

Sometimes nbsphinx does not report properly RST conversion errors (see bug). As a hacky workaround, you might take the nbsphinx.py from ~/.local/lib/python3.5/site-packages/ , make a copy of it in your project home and patch it like this When you call sphinx, it will generate RST files in _build/jupman-rst/.

Of course, things can be cleaner using a virtual env with venv

Git performance notes

Current suggested setup for hosting on Github is creating branch gh-pages and using Github Actions to populate it with html, zips, pdf and epub files. While keeping all that stuff versioned may seem pretty inefficient, apparently git is pretty good at compressing binary files

The size of .git repo for a 1000 pdf page project SoftPython with 300 commits and 100 MB of code is:

.git: 183 MB

By truncating gh-pages to last commit and garbage collecting, we get:

.git: 139 MB

If we completely remove gh-pages branch, we get:

.git: 68.7 MB

So gh-pages size is:

one commit: 70.3 MB
300 commits: 114.3 MB

which is not even double than source code git size.

If the repo gets really huge, in order to to shrink it some git knowledge is required.

If the repo is served from another server and you want to truncate that server git repo:

On that server console:

first make sure you are on gh-pages branch:

git checkout gh-pages

truncates previous commits:
```
git fetch --depth=1 origin gh-pages
```
removes various links around which may still point to old commits:

git reflog expire --expire-unreachable=now --all

actually deletes from disk old commits:

git gc --aggressive --prune=all

Note the result of truncation cannot be pushed back to origin as git would complain it is a shallow branch.

[ ]:

Exams management

Jupman comes with a script to manage exams called exam.py, which allows to manage the full cycle of an exam.

What is an exam

Exam text is represented as Jupyter notebooks, which are taken from _templates/exam/solutions/exam-yyyy-mm-dd.ipynb

Exercises for students: they are supposed to be the exam notebook itself and / or plain python files (or the notebook itself) plus unittests and relative solutions.

Marks spreadsheet: By default there is also an LibreOffice spreadsheet to give marks, in case you need it.

When you initialize an exam with the init command, for example for date 2000-12-31, all the presets in _templates/exam/ are copied to private/2000-12-31/ and private/2000-12-31/solutions. Presets can be changed at will to suit your needs. When packaging, student zip is assembled in private/2000-12-31/student-zip

System is flexible enough so you can privately work on next exams in private/ folder and still being able to publish modifications to main website. After an exam, you can copy the private exam to the public folders in past-exams/.

Exam commands

To see the help:

python3 exam.py -h

To see help for a particular subcommand, like i.e. init, type the subcommand followed by -h :

python3 exam.py init -h

Running commands should be quite self-explanatory.

NOTE: as of today (Dec 2019) software may contain bugs, but at least we check for major misuses (like trying to overwrite existing exams).

In the file create-exam-example.sh there is a typical run of the script, which creates the example exam for date 2000-12-31. Notice it might ask you to delete the existing 2000-12-31 exam, if it does just follow the instructions. Here’s the output:

> ./create-exam-example.sh
python3 exam.py init 2000-12-31
  Detected release from git: 3.2.0-3-g30a995c
No GOOGLE_ANALYTICS environment variable was found, skipping it

  You can now edit Python solutions, tests, exercises and exam notebook here  :

     _private/2000-12-31/solutions

  DONE.

python3 exam.py package 2000-12-31
  Detected release from git: 3.2.0-3-g30a995c
No GOOGLE_ANALYTICS environment variable was found, skipping it
  Cleaning _private/2000-12-31/server/jupman  ...
  Copying exercises to _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/
  Copying code
    from  _private/2000-12-31/solutions
    to    _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/
    Writing (patched) _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/exam-2000-12-31.ipynb
    Generating _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/trees.py
    Writing _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/example.txt
    Generating _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/lists.py
    Writing (patched) _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/trees_test.py
    Writing (patched) _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/lists_test.py
  Creating dir _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/img
    Writing _private/2000-12-31/student-zip/jupman-2000-12-31-FIRSTNAME-LASTNAME-ID/img/mountains.jpg
  Building pdf ..
  Creating student exercises zip:  _private/2000-12-31/server/jupman-2000-12-31-exam.zip
  Writing jupman.py
  Writing my_lib.py
  Writing _static/img/cc-by.png
  Writing _static/js/jupman.js
  Writing _static/css/jupman.css
  Writing _static/js/toc.js
  Writing _static/js/pytutor-embed.bundle.min.js
  Wrote _private/2000-12-31/server/jupman-2000-12-31-exam.zip

  DONE.


------- Simulating some shipped exams...
mkdir -p _private/2000-12-31/shipped/john-doe-112233
cp _templates/exam/solutions/lists_sol.py _templates/exam/solutions/lists_test.py _templates/exam/solutions/trees_sol.py _templates/exam/solutions/trees_test.py _private/2000-12-31/shipped/john-doe-112233
mkdir -p _private/2000-12-31/shipped/jane-doe-445566
cp _templates/exam/solutions/lists_sol.py _templates/exam/solutions/lists_test.py _templates/exam/solutions/trees_sol.py _templates/exam/solutions/trees_test.py _private/2000-12-31/shipped/jane-doe-445566
------- Done with shipped exams simulation, time to grade ...

python3 exam.py grade 2000-12-31
  Detected release from git: 3.2.0-3-g30a995c
No GOOGLE_ANALYTICS environment variable was found, skipping it
  Copying Python files to execute and eventually grade in _private/2000-12-31/graded/john-doe-112233/graded
  Copying original shipped files (don't touch them!) in _private/2000-12-31/graded/john-doe-112233/shipped
  Copying Python files to execute and eventually grade in _private/2000-12-31/graded/jane-doe-445566/graded
  Copying original shipped files (don't touch them!) in _private/2000-12-31/graded/jane-doe-445566/shipped

  DONE.

python3 exam.py zip-grades 2000-12-31
  Detected release from git: 3.2.0-3-g30a995c
No GOOGLE_ANALYTICS environment variable was found, skipping it

  You can now find zips to send to students in _private/2000-12-31/graded


  DONE.

python3 exam.py publish 2000-12-31
  Detected release from git: 3.2.0-3-g30a995c
No GOOGLE_ANALYTICS environment variable was found, skipping it
  Copying solutions to exams/2000-12-31/solutions
  Copying exam PDF text

  Exam Python files copied.

  You can now manually build and run the following git instructions to publish the exam.
    ./build.py
    git status  # just to check everything is ok
    git add .
    git commit -m 'published 2000-12-31 exam'
    git push


  DONE.


  Finished example exam run !!

[ ]:

Chapter examples

Python example

Download exercises zip 

Browse files online

Example of notebook for exercises in Python files

What to do

unzip exercises in a folder, you should get something like this:

python-example
   python-example.ipynb
   lab1.py
   lab1_test.py
   lab1_sol.py
   lab2.py
   lab2_test.py
   lab2_sol.py
   jupman.py
   my_lib.py

open the editor of your choice (for example Visual Studio Code, Spyder or PyCharme), you will edit the files lab1.py and lab2.py
Go on reading this notebook, and follow instuctions inside.

Let’s begin

You are going to program a simulator of bouncing clowns. To do so, we are going to load this module:

[2]:

import local

[3]:

local.gimme(5)

It was a 5 indeed

Download test data

Local file:

Global image

cc-by-2313

Local exercise image

leaves-983423

Python tutor

[4]:

x = [1,2,3]
y = 6

jupman.pytut()

[4]:

Visualization provided by Python Tutor

[5]:

y = [1,2,3]

jupman.pytut()

[5]:

Visualization provided by Python Tutor

Start editing lab1.py in text editor

[6]:

from lab1_sol import *

add

Implement add function:

[7]:

add(3,5)

[7]:

sub

Implement sub function

[8]:

sub(7,4)

[8]:

[ ]:

Jupyter example

Download exercises zip 

Browse files online

Example of notebook for exercises in Jupyter files.

For python files based example and more, see Python example

What to do

unzip exercises in a folder, you should get something like this:

jupyter-example
     jupyter-example.ipynb
     jupyter-example-sol.ipynb
     jupman.py
     my_lib.py

WARNING: to correctly visualize the notebook, it MUST be in an unzipped folder !

open Jupyter Notebook from that folder. Two things should open, first a console and then browser. The browser should show a file list: navigate the list and open the notebook jupyter-example/jupyter-example.ipynb
Go on reading that notebook, and follow instuctions inside.

Shortcut keys:

to execute Python code inside a Jupyter cell, press Control + Enter
to execute Python code inside a Jupyter cell AND select next cell, press Shift + Enter
to execute Python code inside a Jupyter cell AND a create a new cell aftwerwards, press Alt + Enter
If the notebooks look stuck, try to select Kernel -> Restart

[2]:

# REMEMBER TO  IMPORT jupman !
# This cell needs to be executed only once, you can usually find it at the beginning of the worksheets

import jupman

[3]:

x = [1,2,3]
y = x
jupman.pytut()

[3]:

Python Tutor visualization

[4]:

y = [1,2,3]
w  = y[0]
jupman.pytut()

[4]:

Python Tutor visualization

Exercise 1

Implement inc function:

Show solution

[5]:

def inc(x):
    raise Exception('TODO IMPLEMENT ME !')

Exercise 2

Implement upper function

Show solution

[6]:

def upper(x):
    raise Exception('TODO IMPLEMENT ME !')

Exercise 3

Note everything after the ‘write here’ comment will be discarded. Note you can put how many spaces you want in the comment

Show solution

[7]:

w = 5

#  write here

Exercise 4

Shows how to completely remove the content of a solution cell (including the solution comment)

EXERCISE: write a function that prints ‘hello’

Show solution

[8]:

Exercise 5

Shows the QUESTION / ANSWER feature. All content in ‘ANSWER:’ cell will be stripped

QUESTION: Describe why iPhone n + 1 is better than iPhone n

Show answer

Conclusion

bla bla

Relative image test, Markdown format:

cc-by-78342

Relative image test, HTML img tag:

cc-by-1243

Relative link test, Markdown format:

Back to index

Relative link test, HTML a tag:

Back to index

Jupyter example with custom js and css

Download exercises zip 

Browse files online

Example of notebook for exercises in Jupyter files which calls jupman.init() for injecting jupman.js and jupman.css while editing in Jupyter.

Note calling init() is not mandatory. See also Common files and Custom-js-and-css.

Exercise 1

Implement inc function:

Show solution

[2]:

def inc(x):
    raise Exception('TODO IMPLEMENT ME !')

Exercise 2

Implement upper function

Show solution

[3]:

def upper(x):
    raise Exception('TODO IMPLEMENT ME !')

Jupyter and Python example

Download exercises zip 

Browse files online

Most complex example of a notebook with exercises both in Jupyter and Python files, and ‘advanced’ features

What to do

unzip exercises in a folder, you should get something like this:

jup-and-py-example
   jup-and-py-example.ipynb
   jup-and-py-example_sol.ipynb
   lab.py
   lab_test.py
   lab_sol.py

open the editor of your choice (for example Visual Studio Code, Spyder or PyCharme), and edit lab.py file
Go on reading this notebook, and follow instuctions inside.

Let’s begin

You are going to program a simulator of bouncing clowns. To do so, we are going to load this module:

[2]:

import local

[3]:

local.gimme(5)

It was a 5 indeed

Download test data

Local file:

Global image

cc-by-5754

Local exercise image

leaves-98231

Python tutor

[4]:

x = 5
y = 6
z = x + y

jupman.pytut()

[4]:

Python Tutor visualization

Exercise in Jupyter

Implement this function:

Show solution

[5]:

def hello(s):
    raise Exception('TODO IMPLEMENT ME !')

hello_db = hello("Guybrush")

hello_db[:10]

[5]:

['hello',
 'Guybrush',
 'hello',
 'Guybrush',
 'hello',
 'Guybrush',
 'hello',
 'Guybrush',
 'hello',
 'Guybrush']

Full expected output is in file expected_output_db.py, if you can’t manage to solve the exercise, as a last resort you can type: from expected_hello_db import * (DO NOT copy-paste file content, it would probably mess Jupyter up)

[6]:

from expected_hello_db import *
expected_hello_db[:10]

[6]:

['hello',
 'Guybrush',
 'hello',
 'Guybrush',
 'hello',
 'Guybrush',
 'hello',
 'Guybrush',
 'hello',
 'Guybrush']

Other example:

[7]:

hello_db2 = hello("Threepwood")

hello_db2[:10]

[7]:

['hello',
 'Threepwood',
 'hello',
 'Threepwood',
 'hello',
 'Threepwood',
 'hello',
 'Threepwood',
 'hello',
 'Threepwood']

Exercise using previous output

Write some code which says hello 3 times using previous functionand

[8]:

print(hello('Guybrush')[:6])

['hello', 'Guybrush', 'hello', 'Guybrush', 'hello', 'Guybrush']

Question in Jupyter

QUESTION: Why learn coding?

Show answer

Exercise in Python

Start editing lab.py in text editor

[9]:

from lab_sol import *

add

Implement add function:

[10]:

add(3,5)

[10]:

sub

Implement sub function

[11]:

sub(7,4)

[11]:

Fine grained purging

This cell input will be completely removed

[13]:

print("This cell output will be completely removed")

[ ]:

Big sub chapter

Big docs example 1

header with long title

sub header with long title

subsub header with long title

header with long title

sub header with long title

header with extra super long title

sub header with extra super long title

sub sub header with extra super long title

header with extra super long title

sub header with extra super long title

header with extra super long title

sub header with extra super long title

header with extra super long title

sub header with extra super long title

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

[ ]:

Big docs example 2

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

[ ]:

Big sub chapter A

Big docs example A1

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

[ ]:

Big docs example A2

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

[ ]:

[ ]:

[ ]:

[ ]:

[ ]:

Big sub chapter B

Big docs example AB1

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

reasonable header

reasonable sub header

reasonable header

reasonable sub header

header with long text

sub header with long text

header with long text

sub header with long text

header with long text

sub header with long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header with extra super long text

sub header with extra super long text

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

header

sub header

[ ]:

Example Challenge

Download exercises zip 

Browse files online

This notebook has no solution!

We published solution on Github only for example purposes, but normally all other files ending in -chal-sol or _chal_sol will be gitignored

[3]:

from example_chal_sol import f
f(3)

[3]:

[4]:

def wow(x):
    raise Exception('TODO IMPLEMENT ME !')

[ ]:

Templates

Changelog

Jupman: A template for online manuals made with Jupyter notebooks.

jupman.softpython.org

July 9th, 2023 - 3.5.7

Added:

Softpython theme eye candy #122
- softpython-theme-textures.css with manually embedded base64 images
- jupman-alert-principle in jupman-web.css as backbone for commandments
logo, favicon #123
explicilty stated all dependencies in requirements-build.txt, added create_env script #82
improved manual

Fixed:

relative paths in cell outputs for zips #119
now using relative js and css imports in jupman.init #117
Python Tutor:
- now shows unnest data structures by default #132
- now shows in cloned cells #126 (changed stable ids #107)
- sets are now correctly displaying with grey header #125
- tutor Visual Studio Code no longer appears with grey text #136

Removed:

_private/README.md

August 11th, 2022 - 3.5.4

Python Tutor:
- added credits #111
- visualized vertical separator bar #110
- fixed red arrow misalignment #105
- stable ids #107
- removed ‘Customize visualization’ #108
Github actions: automatically build themed version #100
SoftPython theme #92: various fixes
Fixed relative html a, img with attributes in markdown not working in zip: #113

June 4th 2022 - 3.5

generated html can be really used offline #96, also fixes wrong math symbols with local build #3
automated testing on github actions #99
virtual env install, pinpointed build depenedencies #82
fixed text overflow on smartphones #94, fixed softpython theme flag
github actions: always reset html output #98

April 28th 2022 - 3.4

fixed softpython theme font size #92
restructured manual

February 25th 2022 - 3.3

Implemented:

jupman-preprocess #64
big docs support #77
Challenges support (suboptimal but usable) #56
jupman-purge #59
jupman.draw_text for #66
jupman.save_py function
jupman.get_doc as nice way to print function documentation #68
jupman.draw_text to show some ASCII characters in local build #66
jupman.mem_limit for Linux #62
Home link should point to index.html #71
optional parameter conf to jmt.init
deterministic zip #60 (requires python 3.7)

Fixed:

Notebook validation failed: Non-unique cell id error #78
exam pdf build breaks when using images #61

Defined:

how to use custom anchors #80
how to have single pages like References at menu bottom #70

October 17th 2020 - 3.2

added optional build on Github Actions
solutions are finally hidden on the website, with a click-to-show button!
introduced generic jupman-togglable and specific jupman-sol CSS classes
improved menu navigation
added softpython theme
images are now shown centered in HTML
moved to jupman.softpython.org
fixed write here tag not preserving the line
deprecated jupman_tools.ignore_spaces in favor of tag_regex
updated nbsphinx to 0.7.1
updated sphinx_rtd_theme to 0.4.3
updated sphinx to 2.3.1
updated pygments to 2.7.1

January 16th 2020 - 3.1

removed jupman.init root parameter
bugfixes
upgraded nbsphinx from 0.3.4 to 0.5.0
upgraded sphinx_rtd_theme from 0.2.5b1 to 0.4.3
upgraded sphinx from 1.7.6 to 2.3.1
upgraded recommonmark from 0.4.0 to 0.6.0

December 29th 2019 - 3.0

much simplified folder structure
- Issue 33
removed solutions from header requirement
- Issue 32
introduced tests (pytest, hypothesis)
removed old_news in favor of changelog.md
Latex:
- much better PDF cover
- using xelatex
- set up unicode mappings
several fixes

September 24th 2018 - 2.0

now using index.ipynb as home. Hurray !

September 19th 2018 - 1.0

fixed build.py
added html templates examples
cleaned toc (was showing too much when loading)

August 26th 2018 - 0.9

implemented generation of exercises from solutions [Issue 14(https://github.com/DavidLeoni/jupman/issues/14)
reverted to old jupman.init() code Issue 12

August 12th 2018 - 0.8

Prepended all functions in jupman.py with jupman_
replaced index with proper homepage. see Issue 11
- from now on you need home.ipynb file, because replacing index.rst is a nightmare!
- new index.rst is just a placeholder which simply redirects to home.html. Do not modify it.
- put the toctree in toc.rst
exercises ipynb can now stay in exercises/ folder; when exercises are zipped, jupman automatically adds to the zip the required site files. see Issue 12
Tried %run at beginning of notebooks, without much satisfaction (see discussion in Issue 12):
disabled toc by default in html files. To enable it, in python use %run -i ../../jupman --toc
renamed past-exams directory from ‘past-exams’ to ‘exams’
created info, error, warn, fatal functions to conf.py
introduced new variable exercise_common_files in conf.py for common files to be zipped
added pages exam-project , markdown , project-ideas,
added cc-by.png
renamed changelog.txt to changelog.md
now using templates with curly brackets in in templating, like _JM_{some_property}
jupman.js : now when manually saving html in Jupyter, resulting html correctly hides cells
Fixes https://github.com/DavidLeoni/jupman/issues/2 : now toc is present in local build for pdfs

August 3rd 2018 - 0.7

added jupman.py pytut() for displaying Python tutor in the cells
added jupman.py toc=False option to jupman.py init to disable toc
removed jupman.pyuseless networkx import from
fixed usage indentation
added changelog.txt

Past Exams

[ ]:

Exam project

**For general (credits, attendance) info, see course description

Delivery times

Ideas for possible projects: See here

Last update: TODO

In short:

What to do

First of all: send by email to TODO@TODO.COM a brief description of the project, to decide what to do. I will create a Google doc to keep track of progresses and / or problems found.

Once the project is defined, go on like this:

1 - Download zip with template (view online files TODO )

After unzipped, you will find a folder named NAME-SURNAME-ID, with these files inside:

- NAME-SURNAME-ID
    - project.ipynb
    - markdown.ipynb
    - requirements.txt
    - img
        - example.png

2 - Rename the folder NAME-SURNAME-ID with your data

3 - run Jupyter from the folder you just renamed

4 - edit file project.ipynb , closely following the indications in the following technical requirements

5 - Once done, send project by email to TODO@TODO.COM

Technical requirements

Write in Markdown

Python code

`requirements.txt` file

Graphical interfaces

Be careful to

Project ideas

TODO

Last update: TODO

Introduction

[ ]:

Jupman Project

PUT:

TITLE

NAME - ID

Bla bla

[ ]:

Markdown

Briefly explain why markdown is so great ..

[ ]:

Tests

Rendering tests

Shows various recommendations and cornercases, see also softpython themed version.

The page Title has one sharp, sections always have two sharps.

Section 1

bla bla

Section 2

Subsections always have three sharps

Subsection 1

bla bla

Subsection 2

bla bla

Python code

[2]:

x = 4
lst = [1,2,4]

Python in Markdown

x = 4
lst = [1,2,4]

Other quotes

I’m quoted with > symbol on multiple lines. Am I readable?

Somebody asked

I'm quoted with **spaces**
on multiple lines
Am I readable?

Generic quote with backticks
Can you read it?
Is it nice?

# bash quote with backticks
cd foo
ls
cat wiz.txt

Download links

Files manually put in _static :

Download trial.odt
Download trial.pdf

Files in arbitrary folder position :

Download requirements.txt

NOTE: download links are messy, see issue 8

Links to HTML

Link to trial.html

Info/Warning Boxes

Inherited from NBSphinx:

Until there is an info/warning extension for Markdown/CommonMark, such boxes can be created by using HTML

elements like the following

For this to work reliably, you should obey the following guidelines:

The class attribute has to be either "alert alert-info" or "alert alert-warning", other values will not be converted correctly.
No further attributes are allowed.
For compatibility with CommonMark, you should add an empty line between the <div> start tag and the beginning of the content.

Note: This is an info, mixed bold!

Note: This is a warn, mixed bold!

Note: This is a long info, plain title, single paragraph