In order to collect data you need to observe or to measure some property. This property is called variable.

Tables

Let’s look at this data set:

13, 11, 10, 14, 14, 11, 9, 7, 22, 20, 24

What conslusion can we draw? It doesn’t mean anything to us.

But if we see table below we can get some clue that this is data about soccer games of German clubs in 2017 and data is in column AS. We still don’t know what data means because we don’t know what column AS means.

But I also have this information:

• Div = League Division
• Date = Match Date (dd/mm/yy)
• HomeTeam = Home Tea
• AwayTeam = Away Team
• FTHG and HG = Full Time Home Team Goals
• FTAG and AG = Full Time Away Team Goals
• FTR and Res = Full Time Result (H=Home Win, D=Draw, A=Away Win)
• HTHG = Half Time Home Team Goals
• HTR = Half Time Result (H=Home Win, D=Draw, A=Away Win)
• HS = Home Team Shots
• AS = Away Team Shots
• B365H = Bet365 home win odds

So now we know that 13, 11, 10, 14, 14, 11, 9, 7, 22, 20, 24 are Shots that Away team made in game. So 13 is shots (to goal) of Leverkusen in game against Bayern Munich on 18/08/17, 11 is number of shots that Ausburg make in game against HAmburg on 19/08/17.

Types of data - variables

A variable is qualitative if it not possible to take a numerical value(Leauge Division, Date,Team name, Full Time Result) A variable is quantitative if it can take numerical value (Full Time Home Team Goals,Away Team Shots). Quantitative variable can be continuous or discrete. Continuous variable can take any value in given range (Bet365 home win odds). Discrete variable has clear steps between possible values Full Time Away Team Goals can’t take value 1.25.

As You can see presenting data in table provides a lot of information. Data in table tells a lot of, but this data is fine-grained. What if we want to know total or average team shots in season this is not right type of presentation.

Mean Median Mode and Range

Let us make some digression and explain what is Mean, Median and Mode.

Data set: 4, 5, 3, 8, 2, 2, 10, 9

Mean(or average) is number that you get when add all data points and divide by the number of data points. (4 + 5 + 3 + 8 + 2 + 2 + 10 +9)/8 = ⁴³⁄₈ = 5.375

mean=sum of data# of data points

Median is the middle number, found by ordering all data points and picking out the one in the middle (or if there are two middle numbers, taking the mean of those two numbers). 4 and 5 (two middle numbers) ⁹⁄₂ = 4.5. For example if number of variables in data set is even 4, 5, 7, 12, 22 then mean is number in middle of ordered data set 7.

Mode is the most frequent number, the number that occurs the highest number of times, in case of our example data set 2. There can be no mode (1,2,3,4) Mode = {}, one mode(1,1,2,3,4) Mode = {1}, or multiple modes(1,1,2,2,3,4,5) Mode = {1, 2} in a dataset.

Range The largest value in the list is 10, and the smallest is 2, so the range is 10 - 2 = 8 (max - min).

Steam-and-leaf

One interesting presentation is steam-and-leaf

Suppose that a height of children in one class of elementary school is measured and data set is:

65, 54, 76, 72, 71, 83, 72, 92, 60, 42, 73, 84, 72, 97, 72, 91, 53, 63, 84, 85

Let us sort data and take the leading digit (10’s digit) as stem, and trailing digits are leafs.

42, 53, 54, 60, 63, 65, 71, 72, 72, 72, 72, 73, 76, 83, 84, 84, 85, 91, 92, 97

4|2 means

5|34 means 53 and 54

If You turn this on side you can see the “shape” of the height distribution and all data is visible (maintained).

This is called ordered steam-and-leaf because leaf are ordered |7|1222236.

Depending of data set stem can be taken as two digits. 164 184 186 175 187

16|4

17|5

18|467

If data set is large stem-and-leaf diagram is not best method of displaying data and you need to use other methods.

Tally

For large data sets of data you may wish to divide the data into groups, called classes.

We can use tally to present data. For example favorite drinks of 62 students can be presented like this.

Histogram

Data is collected from big food shoop from all types of cerelars and amout of sodium is in cerelars is variable. Data is presented in next table as grouped frequency distribution. [0-49], [50-99] … are classes

There is no clear rule about how many classes should be chosen or what size they should be, but it is usual to have 5-10 classes.

Grouping data into classes means losing some information. We can see in table that there are 5 types of cereals with amount of sodium between 50 and 99 but we don’t know exact values of observations.

Amount of sodium is continuous variable and amount is rounded to the nearest mg. Class labeled 50-99 would contain values from 49.5 up to (but not including) 99.5. These real endpoints 49.5 and 99.5 are referred as class boundaries. 49.5 <= Class 50-99 < 99.5 or [49.5-99.5)

When a grouped frequency distribution contains continuous data, one form of display is the histogram.

Histogram looks similar to a bar chart.

A chart which represents continuous data is a histogram if - the bars have no space between them (they mey be bars of height zero) - the area of each bar is proportional to the frequency

widht-of-class * height = frequency

height = frequency / widht-of-class

Height is known as frequency density

Time is up, for this post :)

Do I want to do this every time I make or change content ? No one sane would like to do that. And rule of thumb is when You have to do something more than three times automate it.

What I want is that every time i push changes to master to build site with hugo and deploy it to programiraj.ba

This is called continuous deployment (CD). If this repository was on bitbucket this could be done using bitbucket pipelines or if I had Jenkins instance installed on AWS. Let us now use CircleCI (and leave bitbucket pipelines for some other tasks in future).

Every CI/CD tool has some kind of pipeline which is usually described with some yaml, json file or by some scripting language that describes process of integration and deployment. Those include some jobs, commands, executors, steps, workflows etc…

Bottom line is that You have to know how to tell CI/CD system what You want to do.

Let’s to describe process in pseudo pipeline steps:

1. When I push new content to github on master branch checkout new code

2. Build site with hugo

3. Publish new site on programiraj.ba (upload via ftp)

For step 1. we have to options pooling or notification (usually via web hooks). Pooling is process where CircleCi (or other CI/CD tool like Jenkins) checks are there any changes on github for every x minutes (or any time period You set), second option is when there is push on master github notifies CircleCi.

Step 2. and 3. are all done on CircleCI.

We could also add more steps to notify by email one or more persons that there is change on site but let’s focus now only on this 3 steps.

CirceCI works like this:

1. Create a directory called .circleci in the root directory of your local GitHub or Bitbucket code repository.

2. Add a config.yml file in the .circleci directory.

We must learn how to write config.yml to do steps 2. and 3.

For this site I use this script

version: 2.1
jobs:
  build-publish-site:
    docker:
    - image: abazovic/hugo:1.0.0
    steps:
    - checkout
    - run:
          name: "Pull Submodules"
          command: |
            git submodule init
            git submodule update --remote
    - run:
        name: "Run Hugo for blog"
        command: cd blog && HUGO_ENV=production hugo -v
    - run:
        name: "Run Hugo for homepage"
        command: cd home && HUGO_ENV=production hugo -v
    - run:
        name: "Deploy website"
        command: deploy/deploy.sh
workflows:
  version: 2
  workflow:
    jobs:
    - build-publish-site

You can find this script on https://github.com/MirzaAbazovic/programiraj.ba and also all code that I use to deploy https://programiraj.ba and https://blog.programiraj.ba

In plain English this script has one workflow that has one job build-publish-site. Job is running on docker image abazovic/hugo:1.0.0 and has steps to: 1. checkout code checkout

1. Pull Submodules Because I use two themes that I added as git submodules.

2. Run Hugo for blog build static site for blog.

3. “Run Hugo for homepage build static site for home page.

4. Deploy website Deploy it (upload files via lftp) using script deploy.sh

What was tricky:

1. Make custom docker image with installed software: hugo and lftp (will describe this in some other post)

2. Had some problems with git submodules, after checkout step I had to explicitly call git submodule init and git submodule update –remote

3. I also had to write script to upload folder with sub folders generated by hugo to remote folder by ftp protocol. Script uses lftp

Script for upload

#!/bin/bash
echo "Upload all files from blog/public to blog.programiraj.ba using lftp"
lftp -e "set ftp:ssl-allow no; mirror -R -P 5  ~/project/blog/public /blog.programiraj.ba; quit" -u $FTP_USERNAME,$FTP_PASSWORD ftp://programiraj.ba
echo "Upload all files from home/public to programiraj.ba using lftp"
lftp -e "set ftp:ssl-allow no; mirror -R -P 5 ~/project/home/public /www; quit" -u $FTP_USERNAME,$FTP_PASSWORD ftp://programiraj.ba

Environment variables $FTP_USERNAME $FTP_PASSWORD are set project on CircleCI.

And that’s it every time I change or add new post I just push it on github and it gets publised on my site.

If You have some questions or suggestions I will gladly answer to You.

Hugo set up is really easy:

1. Download binary for Your OS https://github.com/gohugoio/hugo/releases add it to PATH. More details and other ways to install can be found on https://gohugo.io/getting-started/installing/

2. Create new site

   hugo new site your-static-site-name.com

   1. Get some theme, I used this theme https://github.com/Track3/hermit. bash cd your-static-site-name.com git submodule add https://github.com/Track3/hermit.git themes/hermit

3. To create new post run

   hugo new posts/post_name.md

   1. Optionally:

   In order to have comments (discussion) available on site Open account on disqus and add Your web site to get disqusShortname, set it in config.toml. More info on https://gohugo.io/content-management/comments/. If You Want to track visitors set googleAnalytics https://gohugo.io/templates/internal/#configure-google-analytics. My example is in config.toml