1 Introduction

Jeremy and Rachel, esteemed founders of fastai, quote Blogging is like Resume, only better in this post. Blogging really is one of the best medium to learn while doing. It helps to communicate and formulate ideas, and share knowledge with your peers and colleagues. Blogging for me addresses provides three main motivations:

  1. It helps in improving communication and writing skills.
  2. It provide medium to showcase technical expertise in programming, data science and other domains.
  3. It is a great medium for self learning and teaching.

With nbdev, fastai library and video courses; Fastai team has contributed some of the best material and tools for learning data science specificaaly and programming in general. The top down learning style , with focus on learning the big picture by execution and programming, is an amazingly useful and productive technique especially for someone who is working full time.

Fastpages is the recent addition in the same tradition of tools made available by FastAI team to new and practicing data scientist to facilitate there learning journey. It capitalizes on nbdev codebases, a python literate programming tool and github actions to enable blogging by jupyter notebook and markdown files.

My objective in this post is to showcase some of the minor changes which I have made which enables me to use entire toolchain of fastpages; to blog with emacs org mode along with notebook, markdown or word format. I also intend to demonstrate how powerful org mode really is ; and why it belongs in a Data Scientist toolbox alongwith the list of tools mentioned above.

2 EMACS ORG-MODE: WHAT? WHY? HOW?

BEFORE CAME JUPYTER, BEFORE CAME INTERNET AND EVEN BEFORE THERE WERE PERSONAL COMPUTERS.THERE WAS SOMETHING CALLED EMACS

2.1 The What?

So what is org-mode? At the very basic level org mode or org format is a text markup defined for structured text editing. It is similar to markdown as it is defined in plaintext, priortizes readability and has an human centric approach. Combination with emacs elisp system makes it insanely powerful. Instead of yet another markup syntax; it becomes a system of text editing which is highly extensible and infinitely programmable. With various open source plugins' it can be used like a literate programming tool like notebooks or frontend for latex report creation or even online presentations.

2.2 The Why?

So why should you be interested in Org Mode .Here is a brief list of things that you can do with it.

2.2.1 Blogging

You can write a blog post like this. Since Emacs is insanely powerful for text editing you can do really fast typing.

2.2.2 FastPages Functionality

Lot of bells and whistle related to boxes and link are enabled using org mode link functionality

  • Links are prepended by a directive
  • When doing html export link get converted into jekyll include statement already defined in the code
  • When exporting to other backends it follows the same structure as normal link for that structure
  1. Youtube Video Link

    You can embed individual video or even playlist

    [[yt:https://youtu.be/5haX95nk02E][New Link Feature Org Mode]]
    

    
    [[yt:https://www.youtube.com/playlist?list=PLxc79l2wpbJYTI5rv2os7OoKQMqxReZpr][The Playlist for Org Mode]]
    

  2. Twitter Card Link
    [[twitter:https://twitter.com/jakevdp/status/1204765621767901185?s=20][Altair v4 release]]
    

  3. Remote Image Link
    [[img:https://www.fast.ai/images/fastai_paper/show_batch.png][Credits: https://www.fast.ai/2020/02/13/fastai-A-Layered-API-for-Deep-Learning/]]
    

    Credits: https://www.fast.ai/2020/02/13/fastai-A-Layered-API-for-Deep-Learning/
    Credits: https://www.fast.ai/2020/02/13/fastai-A-Layered-API-for-Deep-Learning/

    [[img:https://upload.wikimedia.org/wikipedia/commons/7/71/ChessPawnSpecialMoves.gif]]
    

    Image

  4. Local Images
    [[img:/images/Emacs.png][Emacs is the king]]
    

    Emacs is the king
    Emacs is the king

    This has been awesome so far.

  5. Boxes and Stuff
    [[alert:This is an alert box]]
    
    [[alert:box][Danger ahead]]
    
    [[info:This is info box]]
    
    [[info:box][This is for your information]]
    
    [[warning:box][You have been warned]]
    
    [[important:box][This is important]]
    
    [[tip:box][You are going to get lucky with emacs]]
    
    [[note:box][This is a note]]
    
    *I am currently in process of fixing doclink functionality inside note box*
    [[note:box][A doc link to [an example website: fast.ai](https://www.fast.ai/) don't work yet.]]
    
    

    This is an alert box

    Danger ahead

    This is info box

    This is for your information

    Warning: You have been warned

    Important: This is important

    Tip: You are going to get lucky with emacs

    Note: This is a note

    I am currently in process of fixing doclink functionality inside note box

    Note: A doc link to [an example website: fast.ai](https://www.fast.ai/) don't work yet.
  6. IFrames and Revealjs Presentation
    [[iframe:https://revealjs.com][Reveal JS Presentation]]
    

    You can embed websites as iframes. This is particularily useful for embedding revealjs presentation like this.

    Infact it is also possible to create revealjs presentation directly from org mode. I will be working on integrating the same in next few days.

    You can also embed powerpoint presentation easily.

2.2.3 Document Embedding

For a longish article, sometimes you would like to split it in different files and combine them together. This workflow is usually better for writing as well; as each document can be focussed on a single aspect of entire topic you want to present in your article.

In org-mode it is trivial to do this using `#+INCLUDE` functionality. As an example above section on Fastpages functionality is actually part of a different blog published in here. I have just imported relevant portions for this post. Code/ configuration files in the bottom section are embedded using the same technique.

#+INCLUDE: "2020-04-29-Awesome-Org-Mode-Links.org::FastPages link" :only-contents t 

2.2.4 Literate Programming

You can do literate programming just like jupyter notebooks, create technical reports, even write full fledged books. Org mode also supports various export backends which means same document can be converted to a html blog, a latex report, beamer or revealjs presentations. Here are a few good links to explore

  • A Multi-Language Computing Environment for Literate programming and Reproducible Research here
  • Org Mode Recipes

  • Literate programming in python with org-mode and noweb here

2.2.5 GTD, Task Manager, Todo

You can use it as task manager, do project management, create to do list. Follow the recommendations of GTD managing your life in a text edit. And many many more things as described here

2.2.6 Are you excited?

What I have done is just touch upon the surface of what is possible with org mode, and by extension blogging on fastpages with org mode. There are many more options and functionality available in org mode including beamer presentation, spreadsheets , agenda views etc… which are awesome but too broad to cover in this introductory blog post. However over time I will continue to post articles on some tidbits' here and there which can greatly enhance your blogging workflow. Now I hope I have motivated you enough, so time to figure out to do it yourself.

2.3 The How?

I have already shown earlier what org mode can do and how it can enhance your writing workflow. Now I will describe how you can do it yourself. Before I go into specifics let me share a bit on general design of fastpages and describe how it can be extended.

2.3.1 Design of fastpages

So here is brief synopsis. Fastpages is based on Jekyll. It already provides you functionality to write a blog in markdown, notebook or word document. This is how it works

  1. Markdown Blogging
    • Markdown is the native format for Jekyll Blogging Engine. Jekyll uses an enhanced version of Markdown called Kramdown which comes with addtional functionality for source code coloring links etc…
    • Layout of Jekyll (Structure of html) is written in `Liquid` template engine. You have liquid templates in two places
      • _layouts folder - used for defining page/post structure.
      • _includes folder - used to define general html blocks or additional functionality which may be included in an article
    • YAML Front Matter - Each article needs to have a yaml frontmatter which defines specifics like title, author, layout to choose, comments to include or exclude , categories , tags etc… which are either generic cross cutting functionality or indicator to select some specific configuration for blogging engine. It needs to come at very top of page
  2. Notebook Blogging
    • Notebook blogging is really what makes fastpages special for data scientist.I should rather specify and say python notebook blogging is what is possible natively in fastpages. Notebook blogging is enabled by a python package called "nbdev" another tool from FastAI.
    • The way it works is as follows
      • You need to specify front matter in first cell of notebook
      • Import / Export/ Hide / Show/ Collapsible hide etc… are directive implemented as comments which need to appear on top of cell to indicate inclusion or exclution from html export
      • In reality notebooks from _notebooks folder are converted to html[not markdown] and exported to _posts folder with a fake (.md) extension as an initial preprocessing step
      • From _post folder Jekyll machinery takes over to convert them into rendered site alongwith markkdown posts
      • Fastpages provides various shims(utlities/methods) to convert some notebook specific directives for links, youtube or boxes to liquid include templates. These are then converted by Jekyll to final version of site.
  3. Word Blogging
    • Word blogging structure is a similar
      • It uses pandoc for "docx" to "html" conversion. Which is copied to _posts folder with (.md) extension
      • Some shim methods are provided by FastPages to handle jekyll _includes and _layout nicely
      • For Words there is only one place to specify "Front Matter". It is _actionfiles/wordfrontmatter.txt file
  4. Automation via Github Actions
    • All processing automation are handled by Github Actions
      • Checkout code to some machine
      • Cleaning up old _site or html
      • Convert files in _notebooks and _word to _posts
      • Building the site using Jekyll into _site
      • Copying / Commiting the _site folder to ghpages etc..
    • Everything in Github Actions are handled by services/ different docker containers running different scripts
    • For local viewing a parallel system is provided using docker-compose.yml and Makefile.

2.3.2 General Idea for extension

So based on the intuition above we basically need to figure out 3 pieces

  • HTML/Markdown Converter :Way to convert custom format to html( with fake .md extension) or actual kramdown style markdown. It needs to have a way to insert/ send YAML FrontMatter to exported files
  • Shims for Jekyll: Some customization scripts to handled Jekyll _include template nicely to manage look and feel
  • Automation Code
    • Github Action [ Include an addtional conversion step in ci.yml or integrate with existing action files possibly in _actionfiles/actionentrypoint.sh ]
    • Update Docker Compose and Makefile

2.3.3 Org Mode Blogging

So with the ideas above , I managed to create some extensions and tricks to enable org mode blogging. It has following parts

  1. Org File Customization

    A small customization is required to export frontmatter from org file.

    
    #+OPTIONS: toc:nil 
    #+BEGIN_EXPORT html
    ---
    layout: post
    categories: [orgmode]
    title: Blogging with Org mode and FastPages
    description: Tutorial to setup Fastpages with org mode
    comments: true
    use_math: false
    ---
    #+END_EXPORT
    
    
    # COMMENT ------------------OPTIONAL FROM HERE------------------------------
    
    
    #+TOC: headline 3
    
    
    • I use org mode specific publishing framework which comes alongwith Emacs.
    • In my export configuration, which I will talk about later, I have some settings which only export body of converted html document.
    • Here first line is for disabling toc specfic to org mode. Default org mode TOC is added on top of body before content which interferes with YAML Frontmatter
    • You can't include #+TITLE directive in org mode , as it is rendered before everything , and thus interfere's with Front Matter.
    • Then comes section for YAML front matter. Wrapping it in html export block allows this to be the first section on the file.
    • Things below OPTIONAL FROM HERE line need not be included. In fact I am still looking forward to customize css so that we can colorize source code similar to notebooks and markdown

    • With these simple rules, you can easily write a blog post with all the functionality of org mode.
  2. FastPages Customization

    I have done minimal non invasive customization to enable automation for org mode in fast pages.

    Note: All the code described below is available in my fork of fastpages at github repo: https://github.com/Rahuketu86/fastpages
    1. Folder Structure

      We need to create a _org folder . Inside _org folder , it should mirror structure of top level directories as follows

      Structure of _org directory in fastpages
      Structure of _org directory in fastpages

      Note: For local images always assume root to be _org [The structure of _org directory should be replica of blog folder structure]. Images will always start with `/images` or `/assets`
    2. Github Action Customization
      • I have created some custom actions, which I save in _customactionsfiles directory at top level. It has got 3 files
        • publish.el - Emacs script to control publishing from _org/posts to _posts folder and converting org to html.

          (require 'package)
          (package-initialize)
          (unless package-archive-contents
            (add-to-list 'package-archives '("org" . "https://orgmode.org/elpa/") t)
            (add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
            (package-refresh-contents))
          (dolist (pkg '(org-plus-contrib htmlize))
            (unless (package-installed-p pkg)
              (package-install pkg)))
          
          (require 'org)
          (require 'ox-rss)
          (require 'ox-publish)
          (require 'ox-html)
          (require 'fpemacs)
          
          (setq org-html-htmlize-output-type nil)
          
          (setq org-publish-project-alist
          	'(
          
              ("org-post-fastpages"
          	    ;; Path to your org files.
          	    :base-directory "./_org/_posts"
          	    :base-extension "org"
          
          	    ;; Path to your Jekyll project.
          	    :publishing-directory "./_posts/"
          	    :recursive t
          	    :publishing-function org-fp-publish-to-html
          	    :html-extension "md"
          	    :body-only t ;; Only export section between <body> </body>
                )
          
          
                ("org-static-fastpages"
          	    :base-directory "./_org/assets"
          	    :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|ogg\\|swf\\|php"
          	    :publishing-directory "./assets/"
          	    :recursive t
          	    :publishing-function org-publish-attachment
          	    )
          
                ("org-images-fastpages"
          	    :base-directory "./_org/images"
          	    :base-extension "css\\|js\\|png\\|jpg\\|gif\\|pdf\\|mp3\\|ogg\\|swf\\|php"
          	    :publishing-directory "./images/"
          	    :recursive t
          	    :publishing-function org-publish-attachment
          	    )
                ("fastpages" :components ("org-post-fastpages" "org-static-fastpages" "org-images-fastpages"))
          
            ))
          
          (defun fastpages-publish-all ()
            "Publish the blog to HTML."
            (interactive)
            (org-publish-all))
          
          
          (provide 'publish)
        • fpemacs.el - Helper functions to enable links functionality translating to html/ jekyll includes. This is really the fastpages shim version of emacs. We can even call it fastpages-emacs library. All the changes should go here

          Warning: This files contains include template in code and can't be rendered fully. Some lines are skipped.

          (defun jekyll-include-local-img (url caption)
            (let ((n_url (s-lex-format "${url}")))
              (if caption
          	  (s-lex-format "<figure>
              <img src=\"${n_url}\"
          	   alt=\"${caption}\">
              <figcaption>${caption}</figcaption>
          </figure>")
          	(s-lex-format "<figure>
              <img src=\"${n_url}\" >
          </figure>"))))
          
          
          (defun embed-img (url caption)
            (cond ((s-starts-with? "/images" url) (jekyll-include-local-img url caption))
          	  ((s-starts-with? "/assets" url) (jekyll-include-local-img url caption))
          	  (t (jekyll-include-remote-img url caption))))
          
          ;;(jekyll-include-img "/images/Emacs.png" "Emacs")
          
          (defun embed-iframe (url)
            (s-lex-format " <div style=\"text-align: center;\">
          	<iframe width=\"560\" height=\"315\" src=\"${url}\" frameborder=\"0\" allow=\"autoplay; encrypted-media\" allowfullscreen></iframe>
             </div>"))
          
          
          (defun get-yt-code (url)
            (car (s-split "&list=" (s-chop-prefixes '("https://www.youtube.com/watch?v=" "https://www.youtube.com/playlist?list=" "https://youtu.be/") url))))
          
          ;;(get-yt-code "https://www.youtube.com/watch?v=SmH3BPpl0TI")
          ;;(get-yt-code "https://www.youtube.com/playlist?list=PLxc79l2wpbJYTI5rv2os7OoKQMqxReZpr")
          ;;(get-yt-code "https://www.youtube.com/watch?v=SzA2YODtgK4&list=PLxc79l2wpbJYTI5rv2os7OoKQMqxReZpr")
          ;;(get-yt-code "https://youtu.be/VawlmG9tsXI")
          
          
          (defun embed-yt(url)
            (if (s-starts-with? "https://www.youtube.com/playlist?list=" url)
          	(let ((code (get-yt-code url))
          	      (embed-base "https://www.youtube.com/embed/videoseries?list="))
          	  (embed-iframe (concat embed-base code)))
              (jekyll-include "youtube.html" (concat "https://youtu.be/" (get-yt-code url)))))
          
          ;;(embed-yt "https://www.youtube.com/watch?v=SmH3BPpl0TI")
          ;;(embed-yt "https://www.youtube.com/playlist?list=PLxc79l2wpbJYTI5rv2os7OoKQMqxReZpr")
          ;;(embed-yt "https://youtu.be/VawlmG9tsXI")
          
          (org-link-set-parameters
           "yt"
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (embed-yt path ))))
           :help-echo "This links helps in exporting link to jekyll youtube liquid template")
          
          (org-link-set-parameters
           "twitter"
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (jekyll-include "twitter.html" path ))))
           :help-echo "This links helps in exporting link to jekyll liquid twitter template")
          
          (org-link-set-parameters
           "img"
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (embed-img  path desc))))
           :help-echo "This links helps in exporting link to jekyll liquid image template")
          
          (org-link-set-parameters
           "alert"
           :face '(:foreground "red" :underline t)
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (jekyll-include-box "alert.html" "text" (or desc path)))))
           :help-echo "This links helps in exporting link to jekyll alert template")
          
          
          (org-link-set-parameters
           "info"
           :face '(:foreground "blue" :underline t)
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (jekyll-include-box "info.html" "text" (or desc path)))))
           :help-echo "This links helps in exporting link to jekyll info template")
          
          
          (org-link-set-parameters
           "warning"
           :face '(:foreground "pink")
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (jekyll-include-box "warning.html" "content" (or desc path)))))
           :help-echo "This links helps in exporting link to jekyll warning template")
          
          
          (org-link-set-parameters
           "important"
           :face '(:foreground "yellow")
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (jekyll-include-box "important.html" "content" (or desc path)))))
           :help-echo "This links helps in exporting link to jekyll important template")
          
          (org-link-set-parameters
           "tip"
           :face '(:foreground "green")
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (jekyll-include-box "tip.html" "content" (or desc path)))))
           :help-echo "This links helps in exporting link to jekyll tip template")
          
          
          (org-link-set-parameters
           "note"
           :face '(:foreground "light blue")
           :export (lambda (path desc backend)
          	     (cond
          	      ((eq 'html backend)
          	       (jekyll-include-box "note.html" "content" (or desc path)))))
           :help-echo "This links helps in exporting link to jekyll note template")
          
            (org-link-set-parameters
            "iframe"
            :export (lambda (path desc backend)
                (cond
          	  ((eq 'html backend)
          	  (embed-iframe path ))))
            :help-echo "This links help in embedding iframe and revealjs presentation")
          
          
          (setq org-html-htmlize-output-type nil)
          
          ;; Define A new backend for fastpages html export
          
          (defun org-fp-code-folding (block)
            (s-lex-format "<div class=\"cell border-box-sizing code_cell rendered\">
              <details class=\"description\">
                <summary class=\"btn btn-sm\" data-open=\"Hide Code\" data-close=\"Show Code\"></summary>
          	<p>
          	   <div class=\"input\">
          		${block}
          	  </div>
          	</p>
              </details>
          </div>"))
          
          (defun org-fp-html-src-block (src-block contents info)
            "Transcode a SRC-BLOCK element from Org to HTML.
          CONTENTS holds the contents of the item.  INFO is a plist holding
          contextual information."
            (if (org-export-read-attribute :attr_html src-block :textarea)
          	(org-html--textarea-block src-block)
              (let ((lang (org-element-property :language src-block))
          	    (caption (org-export-get-caption src-block))
          	    (code (org-html-format-code src-block info))
          	    (label (let ((lbl (and (org-element-property :name src-block)
          				   (org-export-get-reference src-block info))))
          		     (if lbl (format " id=\"%s\"" lbl) ""))))
          	(org-fp-code-folding
          	(if (not lang) (format "<pre class=\"example\"%s>\n%s</pre>" label code)
          	  (format
          	   "<div class=\"org-src-container\">\n%s%s\n</div>"
          	   (if (not caption) ""
          	     (format "<label class=\"org-src-name\">%s</label>"
          		     (org-export-data caption info)))
          	   (jekyll-highlight lang code)))))))
          
          ;;	 (format "\n<pre class=\"src src-%s\"%s>%s</pre>" lang label code))))))
          
          (defun org-fp-inline-src-block (inline-src-block _contents info)
            "Transcode an INLINE-SRC-BLOCK element from Org to HTML.
            CONTENTS holds the contents of the item.  INFO is a plist holding
            contextual information."
            (let* ((lang (org-element-property :language inline-src-block))
          	   (code (org-html-fontify-code
          		  (org-element-property :value inline-src-block)
          		  lang))
          	   (label
          	    (let ((lbl (and (org-element-property :name inline-src-block)
          			    (org-export-get-reference inline-src-block info))))
          	      (if (not lbl) "" (format " id=\"%s\"" lbl)))))
              (jekyll-highlight lang code)))
          
          
          
          (org-export-define-derived-backend 'fastpages 'html
            :menu-entry
            '(?f "FastPages Export Backend"
          	 ((?A "As HTML Buffer (Fastpages)" org-fp-export-as-html)
          	  (?a "As HTML file (Fastpages)" org-fp-export-to-html)))
            :translate-alist '((inline-src-block . org-fp-inline-src-block)
          		       (src-block . org-fp-html-src-block)))
          
          
          
          ;;;###autoload
          (defun org-fp-export-as-html
            (&optional async subtreep visible-only body-only ext-plist)
          
            (interactive)
            (org-export-to-buffer 'fastpages "*Org FP HTML Export*"
              async subtreep visible-only body-only ext-plist
              (lambda () (set-auto-mode t))))
          
          ;;;###autoload
          (defun org-fp-convert-region-to-html ()
          
            (interactive)
            (org-export-replace-region-by 'fastpages))
          
          ;;;###autoload
          (defun org-fp-export-to-html
            (&optional async subtreep visible-only body-only ext-plist)
          
          
            (interactive)
            (let* ((extension (concat
          		       (when (> (length org-html-extension) 0) ".")
          		       (or (plist-get ext-plist :html-extension)
          			   org-html-extension
          			   "html")))
          	   (file (org-export-output-file-name extension subtreep))
          	   (org-export-coding-system org-html-coding-system))
              (org-export-to-file 'fastpages file
          	async subtreep visible-only body-only ext-plist)))
          
          ;;;###autoload
          (defun org-fp-publish-to-html (plist filename pub-dir)
          
            (org-publish-org-to 'fastpages filename
          			(concat (when (> (length org-html-extension) 0) ".")
          				(or (plist-get plist :html-extension)
          				    org-html-extension
          				    "html"))
          			plist pub-dir))
          
          (provide 'fpemacs)
        • custom.sh - Used to run emacs in batch mode with above files

          #!/bin/sh
          
          echo "Inside custom converter code"
          # emacs --batch --no-init-file --load ./_custom_action_files/publish.el --funcall toggle-debug-on-error --funcall fastpages-publish-all
          rm -rf /root/.org-timestamps
          emacs --batch --no-init-file --load ./_custom_action_files/fpemacs.el --load ./_custom_action_files/publish.el --funcall toggle-debug-on-error --funcall fastpages-publish-all
      • Additionally I have defined a public docker image encapsulating emacs and other dependencies for add on which is available from dockerhub as rahuketu86/fastpages-emacs. I add an addtional step to convert org files in my ci.yaml to enable this conversion automatically on git push. My modified ci now looks as follows

        name: CI
        on:
          push:
            branches:
              - master # need to filter here so we only deploy when there is a push to master
          # no filters on pull requests, so intentionally left blank
          pull_request:
            branches:
              - '!cms**'
        
        jobs:     
          build-site:
            if: ( github.event.commits[0].message != 'Initial commit' ) || github.run_number > 1
            runs-on: ubuntu-latest
            steps:
        
            - name: Copy Repository Contents
              uses: actions/checkout@master
              with:
        	persist-credentials: false
        
            - name: setup directories for Jekyll build
              run: |
        	rm -rf _site
        	sudo chmod -R 777 .
        
            - name: convert org files
              uses: docker://rahuketu86/fastpages-emacs
              if: github.event.deleted == false
              with:
        	args: ./_custom_action_files/custom.sh
        
            - name: convert notebooks and word docs to posts
              uses: ./_action_files
        
            - name: Jekyll build
              uses: docker://hamelsmu/fastpages-jekyll
              with:
        	args: bash -c "gem install bundler && jekyll build -V --strict_front_matter --trace"
              env:
        	JEKYLL_ENV: 'production'
        
            - name: copy CNAME file into _site if CNAME exists
              run: |
        	sudo chmod -R 777 _site/
        	cp CNAME _site/ 2>/dev/null || :
        
            - name: Deploy
              if: github.event_name	== 'push'
              uses: peaceiris/actions-gh-pages@v3
              with:
        	deploy_key: $
        	publish_dir: ./_site
    3. Finally for local viewing
      • I had to update docker-compose.yml to include an orgconverter service to use my rahuketu86/fastpages-emacs

        version: "3"
        services:
          fastpages: &fastpages
            working_dir: /data
            environment:
        	- INPUT_BOOL_SAVE_MARKDOWN=false
            build:
              context: ./_action_files
              dockerfile: ./Dockerfile
            image: fastpages-dev
            logging:
              driver: json-file
              options:
        	max-size: 50m
            stdin_open: true
            tty: true
            volumes:
              - .:/data/
        
          orgconverter:
            working_dir: /data
            image: rahuketu86/fastpages-emacs
            volumes:
              -  .:/data/
            command: _custom_action_files/custom.sh
        
        
          converter:
            <<: *fastpages
            command: /fastpages/action_entrypoint.sh
        
          watcher:
            <<: *fastpages
            command: watchmedo shell-command --command /fastpages/action_entrypoint.sh --pattern *.ipynb --recursive --drop
        
          jekyll:
            working_dir: /data
            image: hamelsmu/fastpages-jekyll
            restart: unless-stopped
            ports:
              - "4000:4000"
            volumes:
              - .:/data/
            command: >
             bash -c "gem install bundler
             && jekyll serve --trace --strict_front_matter"
      • Update Makefile to add a single line in convert [# convert word & nb without Jekyll services] to run orgconverter alongwith nbdev converter

        help:
        	cat Makefile
        
        # start (or restart) the services
        server: .FORCE
        	docker-compose down --remove-orphans || true;
        	docker-compose up
        
        # start (or restart) the services in detached mode
        server-detached: .FORCE
        	docker-compose down || true;
        	docker-compose up -d
        
        # build or rebuild the services WITHOUT cache
        build: .FORCE
        	chmod 777 Gemfile.lock
        	docker-compose stop || true; docker-compose rm || true;
        	docker build --no-cache -t hamelsmu/fastpages-nbdev -f _action_files/fastpages-nbdev.Dockerfile .
        	docker build --no-cache -t hamelsmu/fastpages-jekyll -f _action_files/fastpages-jekyll.Dockerfile .
        	docker-compose build --force-rm --no-cache
        
        # rebuild the services WITH cache
        quick-build: .FORCE
        	docker-compose stop || true;
        	docker build -t hamelsmu/fastpages-nbdev -f _action_files/fastpages-nbdev.Dockerfile .
        	docker build -t hamelsmu/fastpages-jekyll -f _action_files/fastpages-jekyll.Dockerfile .
        	docker-compose build 
        
        # convert word & nb without Jekyll services
        convert: .FORCE
        	docker-compose up orgconverter
        	docker-compose up converter
        
        # stop all containers
        stop: .FORCE
        	docker-compose stop
        	docker ps | grep fastpages | awk '{print $1}' | xargs docker stop
        
        # remove all containers
        remove: .FORCE
        	docker-compose stop  || true; docker-compose rm || true;
        
        # get shell inside the notebook converter service (Must already be running)
        bash-nb: .FORCE
        	docker-compose exec watcher /bin/bash
        
        # get shell inside jekyll service (Must already be running)
        bash-jekyll: .FORCE
        	docker-compose exec jekyll /bin/bash
        
        # restart just the Jekyll server
        restart-jekyll: .FORCE
        	docker-compose restart jekyll
        
        .FORCE:

2.3.4 Trying it yourself

Tip: If you quickly want to try above setup yourself. Please follow following instruction
  • If you are starting from scratch. Use my fork of fastpages as template. Instructions are documented here. Please click on my fastpages fork here to start generating your repository from template.
  • After a few seconds you will get an automated pull request. Please setup SSHDEPLOYKEY (secret) and fastpages-key as instructed and merge pull request. Your blog will be deployed and working in few seconds automatically.

Warning: Emacs crowd, please don't create repository with dashes in names

Although I will track the changes in fastai master which may be ahead of my fork. So in case if you have a pre-existing blog or if you would like to have the latest fastpages commits; follow these instructions:-

  • Create a Repository by using fastpages template repository as documented here Please click on here to start generating your repository from template.
  • After you finish the setup process by generating and storing ssh keys, clone your blog repository to local computer.
  • Clone or copy My fastpages fork to a different folder.
  • Copy _org folder from FastPagesDev repository to your blog repository
  • Copy _customactions folder from FastPagesDev repository to your blog repository
  • Copy Makefile, docker-compose.yml and .github/ci.yml to your blog repository. This will overwrite the files in your blog repository.
  • Now you are ready. You can either build your repository locally or commit your code and see it working on published gh-pages

3 Conclusion and way forward

3.1 Org Mode Bridge and Other systems

Org mode on emacs is a really powerful tool for text processing, writing and literate programming. With all the development done on this repository; it is now fairly easy to get started with org mode on fastpages. In future similar techniques can be adopted to create exporter for Rmarkdown or literate-julia or any other target. In view of author, it is important to use the language native to programming system to build these extensions. So for Rmarkdown , it would be useful to leverage existing infrastructure in "knitr" and "bookdown/ blogdown" packages rather than building things from scratch or extending nbdev. This is less burdensome to maintain and enables a writing flow where we don't have to go back and forth from existing system.

Now even though lot of functionality is enabled in orgmode bridge to fastpages, which mirrors or sometimes extends notebook version there are a few pending issues

3.2 Outstanding Tasks

  1. TODO Fix links inside boxes in org mode
  2. TODO Provide more controls on code folding , currently implemented on every block.
  3. TODO Define a specific layout for org files which can include some common header files.
  4. DONE Define a mechanism for code highlighting
    • It can be done using custom css like org.css. But thing approach need some customization to match with default styles
    • Another approach is to use highlight.js and minor customization to read code blocks. This may be more reliable way to approach this

    Note: Best Solution was to define a derived export backend and wrap source block in highlighting liquid template. It is now implemented in fpemacs.el
  5. TODO Handle internal links and html export more robustly
  6. TODO Enable org-reveal functionality to directly generate and intergrate reveal-js presentation in blog post
  7. TODO Create a demo for exporting to beamer and intergrating with blog post

    Tip: Some custom function to streamline above might need to be included in fpemacs.el

    I hope you find this blog post informative to get started with org mode.Please share your suggestions below which can help in improving or extending this setup.