A simple portfolio website built with Flask and JavaScript. The idea is that it integrates with the GitLab Projects API to populate cards on the "projects" page of the site.
The README.md is available for reading via a button which displays a popup modal box (fullscreen), or the user can click the link that directs to the GitLab repository.
Since GitLab Flavoured Markdown (GFM) supports all sorts of extensions such as fenced code and highlighting, LaTeX style maths notation and TOC's, the projects can also take the form of an article.
Perhaps most importantly, it supports darkmode theme, which is enabled by default. Theme selection is remembered with the use of a cookie.
I built this site to utilise YAML and Jinja for a lot of the customisation and configuration, for the added ease of updates.
The "branding" can be update via the static/yaml/branding.yaml
:
name: Shiva Hamid
titles:
 Creator
 Destroyer
links:
gitlab: https://gitlab.com/shivahamid
linkedin: https://linkedin.com/in/shivahamid
twitter: https://twitter.comshivahamid
The "about" page paragraph/s can be update via the static/yaml/about.yaml
file:
about:
 This is a paragraph. It has words. The words in each list item are put into a single <p> element.
 However, you can also add HTML tags to the words here, <h2>like this</h2>.
 You can even add links, just check out <a target="_blank" href="https://shanehull.com/about">this</a>.
The template loops through these items using Jinja, and adds a <p>
tag per YAML list item to the page.
Projects are pulled directly from GitLab using the GitLab API, which is scheduled to update every 10 minutes using FlaskSPScheduler.
To enable this, an API token is needed with read_user
permissions, which can be obtained by following this guide.
The token is placed (along with the relevant GitLab URL, gtag for analytics and a debug flag) in a new config.yaml
file:
gitlabtoken: w4egW4yq45y3q3t4s
gitlaburl: https://gitlab.com
gtag: GLBY82GDSRA
debug: False
The static/yaml/projects.yaml
file is then populated with GitLab projects paths and project ID's, like so:
flaskportfolio:
gitlabid: 28830359
anotherrepository:
gitlabid: 28710215
The key in the YAML file must match the GitLab path, or it will error out and skip that project.
Projects are sorted first by latest activity, then by date created.
The project modals contain rendered HTML for the associated README.md file.
I'm using pythonmarkdown with extensions to support:
[TOC]
(currently supported) and GFM's [[__TOC__]]
This is a simple script that will subscribe to The Things Network v3 (via MQTT) and write the payload values received to a PostgreSQL database.
It can handle multiple devices, managed via a config file. The script subscribes via MQTT and creates a table per device, then writes the payload data as it comes in.
It provides support for the TimescaleDB, an extension built to optimise PostgreSQL for timeseries data.
If you want to use the TimescaleDB extension, it needs to be installed and enabled by following the howto guide.
If you don't just set hypertables: False
in the config file.
The advantage of TimescaleDB is a huge improvement in performance and functionality for seriesdata. Basically, we can cram all of our readings for a particular device into the same row, against the same timestamp, without the need for any JOINs and without the risk of it bogging down our DB when it gets large. Hypertables are partitioned into "chunks" and interlinked by a column timestamp to optimise them for timeseries operations. We also get some extra timebased querying functions to make data visualisation and manipulation more powerful.
For a comparison of the performance of TimescaleDB vs vanilla PostgreSQL, check here.
The configuration file path is specified with a CLI argument.
usage: ttn2postgresql.py [h] [c CONFIG]
example:
python3 ttn2postgresql.py c /etc/ttn2postgresql/config.yaml
python3 ttn2postgresql.py config /etc/ttn2postgresql/config.yaml
This section requires an API key (referred to as the password here, since that's what MQTT calls it), which can be specific to the app or global to your account.
To get an API key, follow this guide.
The server address and port can be found on the same page. You should use port 8883 for TLS support. I haven't tested the nonTLS port.
ttn:
app_id: "gpstrackersit123"
password: "NNSXS.BLAH5467GRDGD4TF3446AW34ET.ODFHR5F7RDGLLVMI2O76DX6VG75SDF3F4WSILZ2CT456JNW"
server: "au1.cloud.thethings.network"
port: 8883
This is fairly straightforward, aside from the hypertables flag.
Hypertables are a TimescaleDB feature, you can read more about them here. This is what enables the tables to be partitioned into "chunks", which provides us with all the performance improvements that TimescaleDB offers.
The database and user must exist, with the correct access configured in your pg_hba.conf
file if your are authenticating externally.
postgres:
username: "ttn"
password: "securepassword!"
host: "localhost"
port: "5432"
database: "ttn"
hypertables: True
Devices are added here and must match the device ID's in the console. As mentioned above, a table is created for each device. The columns specified in the config below will be created dynamically for each device table on first run. If you screw this up, drop the table and start again  it won't recreate the table if it already exists.
The data types must match the generic types specified in SQLAlchemy's documentation here.
devices:
gpstracker0:
columns:
analog_in_1: Float
altitude: Float
latitude: Float
longitude: Float
gpstracker1:
columns:
analog_in_1: Float
altitude: Float
latitude: Float
longitude: Float
To run the script as a service with systemd, create a user:
$ sudo useradd r d /opt/ttn2postgresql ttn
Clone the repo and move it to an appropriate directory:
$ git clone https://gitlab.com/shed909/ttn2postgresql
sudo mv ttn2postgresql /opt/
Install requirements:
$ cd /opt/ttn2postgresql
$ pip3 install r requirements.txt
```
NOTE:
If you get an issue while installing psycopg2, try install the following:
```bash
$ sudo aptget install libpqdev pythondev
Create a config file:
sudo vim /etc/ttn2postgresql/config.yaml
ttn:
app_id: "gpstracker"
password: "NNSXS.BLAHYKHWAKGWEJYSJ6EUTTJYA8EUVHCZLDQ5LDI.37FGIYSNATSKLT9EEJMU9682U3R63DQEFOIPYRWSYLD2WTF492UY"
server: "au1.cloud.thethings.network"
port: 8883
postgres:
username: "ttn"
password: "securepassword!"
host: "localhost"
port: "5432"
database: "ttn"
hypertables: True
devices:
gpstracker0:
columns:
analog_in_1: Float
altitude: Float
latitude: Float
longitude: Float
gpstracker1:
columns:
analog_in_1: Float
altitude: Float
latitude: Float
longitude: Float
Create a service file:
sudo vim /etc/systemd/system/ttn2postgresql.service
[Service]
Type=simple
User=ttn
ExecStart=/usr/bin/python3 /opt/ttn2postgresql/ttn2postgresql.py c /etc/ttn2postgresql/config.yaml
Restart=always
[Install]
WantedBy=multiuser.target
Enable and start the service:
$ sudo systemctl enable ttn2postgresql
$ sudo systemctl start ttn2postgresql
See the example Dockerfile, which uses the official Python3 image to dockerize the script.
A dockercompose example of running ttn2postgresql with a TimescaleDB container is also available in the dockercompose.yaml file.
To troubleshoot issues, set the logging level to DEBUG via the provided CLI argument.
Example run command:
$ /usr/bin/python3 /opt/ttn2postgresql/ttn2postgresql.py c /etc/ttn2postgresql/config.yaml l DEBUG
The RSA (Rivestâ€“Shamirâ€“Adleman) algorithm was named after the 3 scientists/mathematicians who invented it in 1977:
It is an algorithm for public key encryption and is one of the oldest algorithms of its type. It aims to create a oneway function that can be used to encrypt and decrypt data between private parties. The data can be transmitted between the two private parties, without the risk of an eavesdropper reading it.
I'll explain the steps we take to generate a public and private key pair using the RSA algorithm, as well as give some practical examples.
We will use small numbers as apposed to extremely large numbers (1024bits+, as seen in the real world), which are 's of digits long and would be far to compliated for a practical example.
It is assumed you have some knowledge of algebra and prime number theory.
Pick 2 prime numbers, and
Find the product of and , which we will call '', eg:
Find (Phi) of using Euler's totient function
Pick a number, '', which is between and and is coprime to and
Find a number , such that using the Extended Euclidean Algorithm
For this example, let's choose two small prime numbers for our values and . We will use these to generate a public and private key pair.
Step 1: Pick 2 prime numbers, and
Step 2: Find the product of and , which we will call '', eg:
Step 3: Find (Phi) of using Euler's totient function
Step 4: Pick a number, '', which is between and and is coprime to and
We can easily pick one by listing the prime factorisation of our value (which is ), then choosing a number between and that does not share any factors in common.
Step 5: Find a number , such that using the Extended Euclidean Algorithm
As an example, we can rely on the Extended Euclidean Algorithm to find the inverse of .
GCD  13 

Or, an much easier approach, we can calculate it using Euler's theorem:
Our inverse of is , and we can test that they are congruent:
Type this in to a calculator and we find that:
Therefore it satisfies our condition of:
Alice' keys are:
Encryption:
Decryption:
Since we'll need two parties participating in this exhange, we will go over the same steps again, using different numbers. These will be Bob's keys.
Feel free to skip over this, and proceed to Part 2.
Step 1: Pick 2 prime numbers, and
Step 2: Find the product of and , which we will call '', eg:
Step 3: Find (Phi) of using Euler's totient function
Step 4: Pick a number, '', which is between and and is coprime to and
Again, we can pick one by listing the prime factorisation of our value for (which is ), then choosing a number between and that does not share any factors in common.
Step 5: Find a number , such that using the Extended Euclidean Algorithm

Let's calculate it using Euler's theorem:
Our inverse of is , and we can test that they are congruent:
Type this in to a calculator and we find that:
Therefore it satisfies our condition of:
Bob's keys are:
Encryption:
Decryption:
In the previous section, we generated a public/private (encryption/decryption) key pair for both Alice and Bob. Both parties now have a decryption (private) exponent , which they will not share publicly, as well as an ecryption (public) exponent . The encryption exponent can be shared publicly for the other party to use for encrypting messages.
Example A (Alice):
Encryption: Decryption:
Example B (Bob):
Encryption: Decryption:
To encrypt a message, the function is as follows, where is our resulting cipher text:
Bob wants to share a code word with Alice in order to execute a super secret plan to overthrow capitalism.
They have chosen the number in private, as this will indicate to Alice to execute plan C.
We will use Alice' public encryption keys to encrypt the message, which are:
Using the RSA function, Bob can encrypt the message like so:
To decrypt a message, the function is as follows, where is the resulting decrypted message:
Alice will decrypt Bob's message using her private keys which she has not published to Bob.
Using the function, he can decrypt the message like so:
Let's try another message using Bob's keys which he has published to Alice.
We'll go with the number .
Bob's encryption keys are:
Using the function, Alice can encrypt the message like so:
Bob receives this message and decrypts it using his private keys ():
Let's try another message using Bob's keys again.
We'll go with the number .
Bob's encryption keys are:
Using the function, Alice can encrypt the message like so:
Bob receives this message and decrypts it using his private keys ():
Uh oh! What happened?
The message can not be larger than , the modulus, which is referred to as the "keysize".
Another downside of a small keysize is that we can easily factor this to obtain the private keys. Suppose we have an eavesdropper who's name is Eve.
To find Bob's private key, all she has to do is calculate:
So we already know $n$ and we can sub this in like so:
Since and are products of , they can't be larger than , so let's find the square root of :
We know and must be smaller than and one must be smaller than the other, so let's try and solve for by going through the prime numbers between and , and see if they are congruent to , where is our candidate of prime numbers.
We now know our value for , and the equation becomes:
To find , we just have to find a prime number such that , where is our candidate.
And there you have it. Eve can now generate a public and private key pair from and , then decrypt any of Bob's messages.
This was easy enough, but as the numbers get larger, the computational power required to factor them grows exponentially.
Let's try and generate some larger keys with a keylength of 64bits, where and are both 20 digits in size.
We can generate these numbers using openssl, with the folling command:
openssl prime generate bits 64
Step 2: Find the product of and , which we will call '', eg:
Step 3: Find (Phi) of using Euler's totient function
Step 4: Pick a number, '', which is between and and is coprime to and
Now I'm not going to try and find the prime factorisation of , as this would take me a very long time.
We can choose a Fermat number, which will always be coprime.
Step 5: Find a number , such that
We're not going to use the Extended Euclidean Algorithm for obvious reasons, so let's calculate it using Euler's theorem or a calculator like dcode.fr's Modular Inverse Calculator:
Test that this is congruent to .
Type this in to a calculator like Wolfram Alpha and we find that:
Therefore it satisfies our condition of:
And we have that:
To encrypt a message, the function is as follows, where is our resulting cipher text:
We'll choose something small like , since we have ridiculously large numbers to start with and we'll punch this into Wolfram Alpha.
To decrypt a message, the function is as follows, where is the resulting decrypted message:
Again, let's punch this in to Wolfram Alpha, and we should find that: