Stefano Chiodinohttps://stefano.chiodino.uk2022-07-20T07:22:21ZStefano Chiodino's blogstefanoHugo: write, deploy, host2022-07-20T07:24:56Zhttps://stefano.chiodino.uk/hugo-host-and-deployment<p>In this port I'll show you how I write my posts, deploy them to the server, process them with <a href="https://gohugo.io" rel="nofollow">Hugo</a> and host them on my <a href="https://m.do.co/c/875cd23a5c97" rel="nofollow">DigitalOcean</a> server.</p>
<p>I wanted this process to be as easy and quick as possible, with as few extra softwares and services as possible. There are a lot of solutions out there like Wercker but I didn't want more things in my way.</p>
<h1 id="writing"><a class="anchor" href="#writing" rel="nofollow">#</a> Writing</h1>
<h2 id="local"><a class="anchor" href="#local" rel="nofollow">#</a> Local</h2>
<p>I write the posts from IntelliJ IDEA, which is my IDE of choice. Screw Sublime, Atom and all the rest! It's quite heavier but is worth it. You get spell checking, quite good live previews with the Multimarkdown plugin (shame the embedded HTML doesn't render), git integration, and in general a powerful IDE.</p>
<p>Just to be clear I like Sublime and Atom, but every time I use them there is something wrong. Right now the terminal in Atom doesn't work for me. There is an open issue for this but in the meantime you get a blinking unmovable cursor staring at you.</p>
<p>I feel that, since Hugo is quite new, the tools haven't caught up with its popularity yet. Intellij is flexible enough to allow you to have shortcuts <a href="https://www.jetbrains.com/help/idea/2016.1/live-templates.html" rel="nofollow">live templates</a> for your shortcodes. E.g.:</p>
<pre class="chroma"><code><span class="line"><span class="ln">1</span><span class="cl"> <span class="nt"><template</span> <span class="na">name=</span><span class="s">".shortcode"</span> <span class="na">value=</span><span class="s">"{{&lt; $SHORTCODE$ &gt;}}"</span> <span class="na">description=</span><span class="s">"A Hugo shortcode"</span> <span class="na">toReformat=</span><span class="s">"false"</span> <span class="na">toShortenFQNames=</span><span class="s">"true"</span><span class="nt">></span>
</span></span><span class="line"><span class="ln">2</span><span class="cl"> <span class="nt"><variable</span> <span class="na">name=</span><span class="s">"SHORTCODE"</span> <span class="na">expression=</span><span class="s">""</span> <span class="na">defaultValue=</span><span class="s">""</span> <span class="na">alwaysStopAt=</span><span class="s">"true"</span> <span class="nt">/></span>
</span></span><span class="line"><span class="ln">3</span><span class="cl"> <span class="nt"><context></span>
</span></span><span class="line"><span class="ln">4</span><span class="cl"> <span class="nt"><option</span> <span class="na">name=</span><span class="s">"OTHER"</span> <span class="na">value=</span><span class="s">"true"</span> <span class="nt">/></span>
</span></span><span class="line"><span class="ln">5</span><span class="cl"> <span class="nt"></context></span>
</span></span><span class="line"><span class="ln">6</span><span class="cl"> <span class="nt"></template></span>
</span></span></code></pre><p>This allows me to type <code>.s</code>, press enter (if the live template <code>.shortcode</code> is selected), type the name of the shortcode and its parameters and press enter to keep typing without having to type too many symbols or having to move your cursor.</p>
<p>I keep switching between IntelliJ and the localhost page on my browser, where my hugo server is showing how the end result will be (just launch "hugo server" from the root folder). If you have the page open you don't even need to refresh it thanks to Hugo's live reload feature (and don't even have to save the file since IntelliJ does it when it loses focus). This is very quick to do, which is particularly nice when you want to make final tweaks to make sure your post will display as you want.</p>
<h2 id="online"><a class="anchor" href="#online" rel="nofollow">#</a> Online</h2>
<p>If you are using Github and want to write from your browser, or you are not at home, you can use <a href="https://prose.io" rel="nofollow">prose.io</a>. It's a software developed for Jekyll, but works just fine with Hugo. You get some nice shortcuts to format your text with markdown in case you are not too familiar with it. It also allows you to upload images which are committed to your repo immediately. I don't find it quite as powerful as IntelliJ but it's a nice option.</p>
<p>You can have a look at <a href="https://github.com/Draga/go-web/blob/master/_prose.yml" rel="nofollow">my prose configuration</a> to get you started. It's important to know that prose.io can only generate content with yaml metadata. You will have to configure Hugo to do the same by adding <code>MetaDataFormat: yaml</code>.</p>
<p>I've looked at many more free online content editor:</p>
<ul>
<li><a href="http://sofish.github.io/pen/" rel="nofollow">sofish</a></li>
<li><a href="http://www.webhook.com/" rel="nofollow">webhook</a></li>
<li><a href="https://www.contentful.com" rel="nofollow">contentful</a></li>
<li><a href="https://prismic.io/" rel="nofollow">prismic</a></li>
<li><a href="https://stackedit.io" rel="nofollow">stackedit</a></li>
<li><a href="http://dillinger.io/" rel="nofollow">dillinger</a></li>
</ul>
<p>I found them actually extremely good...just not very compatible with Hugo.</p>
<p>Unfortunately I couldn't find any online resource to write and preview Hugo content, so editing content locally and preview it with Hugo server is still the best option for me. I reckon you could host a test environment to preview your draft, deploying with a different branch, by adding <code>-buildDrafts</code> to your <code>hugo</code> or <code>hugo server</code> commands.</p>
<p>Hopefully some tool that works perfectly with Hugo will come out soon. Spf13, the creator of Hugo, <a href="https://discuss.gohugo.io/t/web-based-editor/155/41" rel="nofollow">is thinking about creating one</a>.</p>
<h1 id="deployment"><a class="anchor" href="#deployment" rel="nofollow">#</a> Deployment</h1>
<p>I can deploy my website in 2 ways:</p>
<ul>
<li>Push you branch to the repo on the server.</li>
<li>Push the master branch to the main repo. In my case this is on Github but can work with pretty much anything else.</li>
</ul>
<p>These both trigger a small script which I've adapted from <a href="https://www.digitalocean.com/community/tutorials/how-to-deploy-a-hugo-site-to-production-with-git-hooks-on-ubuntu-14-04" rel="nofollow">this tutorial on Digitalocean</a>.</p>
<pre class="chroma"><code><span class="line"><span class="ln"> 1</span><span class="cl"><span class="cp">#!/bin/bash
</span></span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="cp"></span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="c1"># Better be specific rather than using $HOME because it wouldn't work with all users.</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="nv">GIT_REPO</span><span class="o">=</span>/home/draga/go-web
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="nv">WORKING_DIRECTORY</span><span class="o">=</span><span class="nv">$GIT_REPO</span>/public
</span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="nv">PUBLIC_WWW</span><span class="o">=</span>/home/draga/public_html
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="nv">BACKUP_WWW</span><span class="o">=</span>/home/draga/backup_html
</span></span><span class="line"><span class="ln"> 8</span><span class="cl">
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="nb">set</span> -e
</span></span><span class="line"><span class="ln">10</span><span class="cl">
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="c1"># Clean working dir.</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl">rm -rf <span class="nv">$WORKING_DIRECTORY</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="c1"># Backup current website. Use -z if moving across network to save bandwidth.</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl">rsync -aq <span class="nv">$PUBLIC_WWW</span>/ <span class="nv">$BACKUP_WWW</span>
</span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="c1"># Restore backup if an error occurs from now on.</span>
</span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="nb">trap</span> <span class="s2">"echo 'A problem occurred. Reverting to backup.'; rsync -aq --del </span><span class="nv">$BACKUP_WWW</span><span class="s2">/ </span><span class="nv">$PUBLIC_WWW</span><span class="s2">; rm -rf </span><span class="nv">$WORKING_DIRECTORY</span><span class="s2">"</span> EXIT
</span></span><span class="line"><span class="ln">17</span><span class="cl">
</span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="c1"># fetch repo to make sure it's up to date if working with remote webhook.</span>
</span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="nb">cd</span> <span class="nv">$GIT_REPO</span>
</span></span><span class="line"><span class="ln">20</span><span class="cl">git fetch
</span></span><span class="line"><span class="ln">21</span><span class="cl">git checkout origin/master
</span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="c1"># Build the website. GO HUGO!</span>
</span></span><span class="line"><span class="ln">23</span><span class="cl">/usr/bin/hugo -s <span class="nv">$GIT_REPO</span> -d <span class="nv">$WORKING_DIRECTORY</span>
</span></span><span class="line"><span class="ln">24</span><span class="cl"><span class="c1"># Mirror the output of Hugo to the directory where the website is hosted.</span>
</span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="c1"># Use -c to copy only files that have changed to avoid messing with webhost caches.</span>
</span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="c1"># Use -z if moving across network to save bandwidth.</span>
</span></span><span class="line"><span class="ln">27</span><span class="cl">rsync -aqc --delete <span class="nv">$WORKING_DIRECTORY</span>/ <span class="nv">$PUBLIC_WWW</span>
</span></span><span class="line"><span class="ln">28</span><span class="cl"><span class="c1"># Ask Google webmaster tools to parse your sitemap.</span>
</span></span><span class="line"><span class="ln">29</span><span class="cl">curl https://google.com/webmasters/sitemaps/ping?sitemap<span class="o">=</span>https://stefano.chiodino.uk/sitemap.xml
</span></span><span class="line"><span class="ln">30</span><span class="cl">curl https://www.bing.com/ping?sitemap<span class="o">=</span>https%3A%2F%2Fstefano.chiodino.uk/sitemap.xml
</span></span><span class="line"><span class="ln">31</span><span class="cl"><span class="c1"># Disable trap.</span>
</span></span><span class="line"><span class="ln">32</span><span class="cl"><span class="nb">trap</span> - EXIT
</span></span></code></pre><p>I used the method described in the tutorial above for a while, but it bothered me to push twice since I had to type the password for my ssh key (I've disabled the password access to the server and can now only be accessed with the key). Also what if I wasn't home or wanted to use prose.io?</p>
<p>Solution: git(hub) webhooks!</p>
<p>Thanks to <a href="https://github.com/adnanh/webhook" rel="nofollow">github.com/adnanh/webhook</a>, little software written in go, you can have you server listen on a port and execute a script when invoked.</p>
<p>I currently have it connecting in https, with a secret key, and executing the script when the master branch is pushed.</p>
<p>Now I can even use Github itself to edit my posts. Pretty neat!</p>
<h1 id="hosting"><a class="anchor" href="#hosting" rel="nofollow">#</a> Hosting</h1>
<p>Now this was the <del>long</del> fun part. I wanted it all so it took a lot of learning, hence the fun.</p>
<p>I heard of nginx a lot and some research showed that it seems to be the go-to solution. Here are some, more performance related, details of how it performs for me: <a href="/content/server-load-test" rel="nofollow">server load test</a>.</p>
<p>I've then enabled the pagespeed module which is very complex and heavy, but definitively worth it!
The website now scores <a href="https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fstefano.chiodino.uk%2F" rel="nofollow">100/100 in Google PageSpeed</a>, both in mobile and desktop!</p>
<p>I score 99/100 in <a href="https://gtmetrix.com/reports/stefano.chiodino.uk/FsGkxV4j" rel="nofollow">GTmetrix</a> just because there is a bug in the pagespeed module (<a href="https://github.com/pagespeed/ngx_pagespeed/issues/1064" rel="nofollow">1064</a>) and the very header is added twice, and for some reason GTmetrix seems to think that is not there...? Or maybe is because pagespeed seems to add it on subsequent reloads. Another small penalty is for the G analytics code being cached for just 2 hours, it's out of my control and don't think is worth self hosting it.</p>
<p>YSlow seems to be mostly concerned of the lack of CDN.</p>
<p>You can find all the scripts that I've used to set up my server on <a href="https://github.com/Draga/serverScripts" rel="nofollow">github</a>. They are ordered in the same way they are in the filesystem, so you should get the idea. There is a script to install nginx with the brotli and pagespeed module, directives to use webhook as a service, the entire nginx configuration, etc.</p>
How to write content, deploy automatically and host your websiteServer load test2022-07-20T07:24:57Zhttps://stefano.chiodino.uk/server-load-test<p>I've recently tried <a href="https://m.do.co/c/875cd23a5c97" rel="nofollow">Digitalocean</a> hosting and decided to host this blog on it. I'm running a $5 / month server, or droplet as they call it, with 1 CPU, 512 MB of ram, 20 GB SSD and 1 TB of bandwidth. You even get $10 credit when you sign up, sweet!</p>
<p>You may look down on a $5 machine but I've found it to be more than enough for my need.</p>
<p>I'm serving "static" content that has been generated by <a href="https://gohugo.io" rel="nofollow">Hugo</a></p>
<p>I recently ran some load testing at work for a newly launched website so I decided to test this $5 server. I've been using <a href="https://loader.io" rel="nofollow">loader.io</a> since its basic usage is free.</p>
<h2 id="the-test"><a class="anchor" href="#the-test" rel="nofollow">#</a> The test</h2>
<p>The server responds to a certain number of clients every second for a full minute.</p>
<p>The majority of the users must receive the page in < 5 secs. Only the html is loaded.</p>
<h2 id="plain-html"><a class="anchor" href="#plain-html" rel="nofollow">#</a> Plain HTML</h2>
<p>10.000 requests a second! (that is the maximum that loaded.io goes on a free plan)</p>
<h2 id="enable-strong-https"><a class="anchor" href="#enable-strong-https" rel="nofollow">#</a> Enable strong HTTPS</h2>
<p><a href="https://www.ssllabs.com/ssltest/analyze.html?d=https%3A%2F%2Fstefano.chiodino.uk" rel="nofollow">How strong you say?</a></p>
<p>As expected enabling https is taking its toll. The server can't accommodate less than half of the requests, "just" 4.000.</p>
<h2 id="full-on-pagespeed-on-strong-https"><a class="anchor" href="#full-on-pagespeed-on-strong-https" rel="nofollow">#</a> Full on PageSpeed on strong HTTPS</h2>
<p>I've then enabled Google's PageSpeed module for Nginx and enabled every performance related filter!</p>
<p>I've enabled 27 extra filters on top of the 34 enabled by default, leaving just 7 out that were just for compatibility or counterproductive, hell I've even enabled the filter to make the G analytics script async when it's already implemented that way.</p>
<p>Result: a quite dramatic 150 clients/sec 😀</p>
<p>Here is the list of filters that I've enabled:</p>
<pre class="chroma"><code><span class="line"><span class="ln"> 1</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">responsive_images</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 2</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">outline_css</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 3</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">outline_javascript</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 4</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">move_css_above_scripts</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 5</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">move_css_to_head</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 6</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">rewrite_style_attributes</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 7</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">prioritize_critical_css</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 8</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">make_google_analytics_async</span><span class="p">;</span>
</span></span><span class="line"><span class="ln"> 9</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">canonicalize_javascript_libraries</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">10</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">inline_google_font_css</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">11</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">local_storage_cache</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">12</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">convert_to_webp_animated</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">13</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">insert_image_dimensions</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">14</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">inline_preview_images</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">15</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">resize_mobile_images</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">16</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">remove_comments</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">17</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">collapse_whitespace</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">18</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">elide_attributes</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">19</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">extend_cache_pdfs</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">20</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">sprite_images</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">21</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">rewrite_domains</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">22</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">trim_urls</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">23</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">remove_quotes</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">24</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">defer_javascript</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">25</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">dedup_inlined_images</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">26</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">lazyload_images</span><span class="p">;</span>
</span></span><span class="line"><span class="ln">27</span><span class="cl"><span class="k">pagespeed</span> <span class="s">EnableFilters</span> <span class="s">insert_dns_prefetch</span><span class="p">;</span>
</span></span></code></pre><h2 id="brotli-max-compression"><a class="anchor" href="#brotli-max-compression" rel="nofollow">#</a> Brotli (max compression)</h2>
<p>Just add Accept-Encoding: br to loader.io request headers and... 75 clients / sec. Not bad considering that I've enabled the maximum (11) level of compression!</p>
<p>..and, if you are curious about the compression, my homepage file sizes are (method from <a href="https://hacks.mozilla.org/2015/11/better-than-gzip-compression-with-brotli" rel="nofollow">Mozilla's brotli post </a>):</p>
<table>
<thead>
<tr>
<th>Algorithm</th>
<th>Size</th>
</tr>
</thead>
<tbody>
<tr>
<td>none</td>
<td>9508</td>
</tr>
<tr>
<td>gzip</td>
<td>3344</td>
</tr>
<tr>
<td>brotli</td>
<td>2732</td>
</tr>
</tbody>
</table>
Load testing a $5 Digitalocean server