Automate local multi-page setup

create projects and v-host settings automatically

clockwork's photo
clockwork
·

13 min read

Automate local multi-page setup

Table of contents

.noise

hello_friend,

I want to start this series with a couple of easy articles aimed at new users. So let's start up with a tutorial on how to set up local dev-environments based on apache2. There are different approaches to achieve this, I saw companies symlink their projects and use a 'wwwhome'-alias to set the basepath. You might also be tempted to just put everything in sub-folders. I want to introduce y'all to a different approach, one that yields true multi-domain environments to start hacking away at.

My main motivation was that I find it cumbersome to set up new pages, often spending more time with configuration than on coding. So I wanted to create a script that is easy to use with one call, and afterward be able to just call my-project.com and get a working environment ready.

If you came here just looking for the script itself, check out my Github.

Here's some music to help you focus while reading the post:

https://www.youtube.com/watch?v=CVUhB7_cmgw&list=PLVU8Wz0hJt01eC5avwlCDd7O-Wmlkm8Mm&index=8

Initial thoughts

The first step is to create a folder structure and pass ownership to www-data. Seems easy enough, however, we need to ask the user what they want their new page to be called. So we need to read user input and store it into memory.

We also should check whether the caller is root ( or obtained a sudo lock ). We need to fiddle around creating files, chowning stuff and writing to files that belong to somebody else ( the www-data user ).

Next, we need to write the files and also check whether the user has SSL set up on their machine. These are quite a bunch of things we have to consider. So here's what we're gonna do: Together we bring the script to a state where you could use it in a professional dev environment, and the rest, like prettifying it or introducing flags, is up to you. ( ͡° ͜ʖ ͡° )

Let's write down the steps for this script:

  1. ask user for domain name

  2. check whether there is already a project with that name

  3. check whether user is root

  4. create the files and folders, pass ownership to apache2 user

  5. create vhost files

  6. enable the page

  7. write hosts file entries

  8. open the folders we worked in so user can doublecheck configuration

  9. restart the server to load the configuration

The way we write shellscript

The first thing I want to clarify is that I am a follower of functional scripting. There are many different approaches on how to write a shell script. Often when I fiddled with my .bashrc, I used aliases. While this is good for small commands you want to inject into the shell ( like a custom ls command ), it starts getting really messy when you try writing multi-command aliases. Or even worse, you start using aliases in alias commands. This ain't the way.

Another approach is to just write a long script, front to back. While I could understand choosing this approach for scripts up to about 20 lines, it is starting to get chaotic when writing anything beyond. So this ain't the way either.

So the ideal solution is to use functions. Let's break this down to an easy-to-understand example here:

#!/bin/bash

say_hello(){
    echo "hello $1"
}

say_hello user
say_hello friend

##########################
# output : 
# hello user
# hello friend
Explanation
If you're familiar with C programming, you might have spotted the only real pitfall here already: You need to define the functions before calling them, since, unlike C, we don't have any definitions going on here, so the shell simply won't know what you want to call.

Getting the basics done

Let's start by doing rather simple tasks:

  • read the desired name

  • check project does not exist already

  • check user privileges

These three points aren't too hard, so let's get them done quickly. Before we start, I need to show you our instance variables. I chose to introduce options via variables rather than flags. Feel free to go ahead and mod the script to read console input, just know that these values probably aren't going to ever change, so you might as well just leave them as they are.

DIR="/var/www/"
SERVERUSER="www-data"
SERVERADMIN_EMAIL="top@kek.mate"
SSL_ACTIVE=true
SSLCertificateFile="/etc/ssl/certs/ssl-cert-snakeoil.pem"
SSLCertificateKeyFile="/etc/ssl/private/ssl-cert-snakeoil.key"
AUTO_ENABLE_PAGE=true
CREATE_LOCAL_HOSTS=true

The boolean options are for optional functionality, so if you aren't going to use SSL, just disable the option right here at the top of the script. Let's move forward to our first real functions.

read_page_name(){

    echo -n "Name for new Page (without www): "
    read PAGENAME
}

is_user_root(){

    if [ $(id -u) -ne 0 ]
    then 
        echo "Please run the script as root!"
        exit 1
    fi
}

check_project_existing(){

    if [ -d "$DIR$PAGENAME/" ]; then
        echo "Error: Project ${PAGENAME} already existing!"
        exit 2
    else
        echo "Installing config files in ${DIR}${PAGENAME}/ ..."
    fi
}

This is a rather simple start, but if you have problems following up, you can read the explanation below. Not everybody writes shell script regularly, but nevertheless, you should have no problems understanding the basic logic of the code.

Explanation variable
By calling read PAGENAME, we introduce a variable called PAGENAME. Since we are doing this before using the variable in the third function, this is fine. If you're ever in a position where you're unsure if a variable is already present, just initialize it right at the start of your script with an empty string. As an important side note, if you are using a variable inside a string, you have to write it like this: ${NAME}, but if you are using it as part of another command f.e., you'd just write $NAME. It is common practice to write variable names in capslock, but you don't have to do it that way.
Explanation if statement
I admit, writing a simple if statement in bash looks pretty ugly. Also, there are different ways of writing said statement, which can lead to further confusion. I decided to use the form with the [ ] square brackets, where you first write a statement in a block like this: $(which bash). This is only necessary for statements that can have multiple different outcomes, like id -u, which is an easy way to check which user ID we have. The -ne after the command is short for not equals. If you ever write assembler code, these are the flags for jumping operations, so f.e. -gt (greater than) will work too. Note that the second if-statement is a short form for the command test, and it only returns true or false, so we shorten the whole statement to -d "$DIR$PAGENAME/".

So we're simply checking whether the user has an id of 0 ( root ) and whether a folder of the chosen name is already present. The DIR variable is simply the default folder of our Apache server, located under /var/www.

Creating the project structure

Next, we are going to prepare the structure for our new project, and this is where we need root privileges the first time. Note that we are calling a second function from inside our first one because these belong together logically, but the function would become too long and confusing if we didn't split it up.

create_sample_page(){

    touch ${DIR}${PAGENAME}/public/index.php
    echo "<html>
    <head>
        <title>$PAGENAME</title>
    </head>
    <body>
        <h1>Success! The $PAGENAME virtual host is working!</h1>
    </body>
</html>" >> ${DIR}${PAGENAME}/public/index.html
}

create_file_structure(){

    mkdir ${DIR}${PAGENAME}
    mkdir ${DIR}${PAGENAME}/public
    mkdir ${DIR}${PAGENAME}/log

    create_sample_page

    chown -R ${SERVERUSER}:${SERVERUSER} ${DIR}${PAGENAME}
    chmod -R 755 ${DIR}${PAGENAME}
    chmod -R 775 ${DIR}${PAGENAME}log

}

As mentioned before, the create_sample_page function needs to be placed above the create_file_structure function so the script can call it successfully.

With create_file_structure, we are going to create folders inside the apache2 directory. Usually, this is /var/www/, but since we can't be sure about this, I used the variable ${DIR}. Note that for some projects, like Laravel, you will need to manually edit these folders again, but for a normal web project, it should suffice.

After we create the folders, we call create_sample_page, where we are just writing a basic index.php page to showcase that everything is working. We are first touching the file ( touch: a Unix command to create a file without context ), then write the whole HTML into the file via the >> pipe redirection.

After we create the folders and the sample page, we need to pass ownership to the user that is associated with our apache2 installation. Per default, this user is called www-data, and you have no real reason to change that, but just in case, I created another variable called ${SERVERUSER} for this. Ownership in Unix systems is structured in the format group : user.

Lastly, we are using chmod to mark the folders and all subfolders ( with -R ) as executable. This is important since otherwise the server won't be able to execute scripts. If you want to know more about this, just follow the link above. Note that many users write it like chmod u+x, but I prefer this pure version ( nods to Razor1911 ).

Creating a VHOST entry for the page

After we create the project setup, we need to activate the page so our Apache will know how to serve it. To achieve this, we must write entries into the sites-available folder of our Apache server. Every apache2 installation has 2 folders located under /etc/apache2/, called sites-available and sites-enabled. As the names suggest, only pages that are listed under sites-enabled will be able to get called by anybody. In general, there is little to no reason to not enable the available pages, and since we are building a development environment, we are going to enable the pages no matter what.

create_vhost_file(){

    touch /etc/apache2/sites-available/${PAGENAME}.conf
    echo "<VirtualHost *:80>

    ServerAdmin $SERVERADMIN_EMAIL

    ServerName $PAGENAME
    ServerAlias www.$PAGENAME

    DocumentRoot $DIR$PAGENAME/public

    <Directory  $DIR$PAGENAME/public>
        Options Indexes FollowSymlinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog $DIR$PAGENAME/log/error.log
    CustomLog $DIR$PAGENAME/log/access.log combined

</VirtualHost>" >> /etc/apache2/sites-available/${PAGENAME}.conf
}
Why not https?
Note that this vhost file is for unsecured http-connections ( port 80 ), but we will also handle secured https-connections in the final script. The reason I'm not showing this here is that you have to install a custom SSL-certificate to use a page with https locally, and this article does not handle doing so. It is, however, pretty easy to install, and Ubuntu even comes with a pre-installed snakeoil certificate.

We are packing all standard information into this Vhost file, again these were swapped out into variables. The standard http port is 80, no need to change that. The DocumentRoot entry is the entry point of the page, so it points to the public folder. It is common practice for web developers to put servable content like pages into a folder called public, while logic is put into a folder called src that resides one layer above.

We also enable the option to follow FollowSymlinks along with other options like require all granted, which in general is necessary for our page to work. Without permission to require external scripts, we can't include classes and libraries in our scripts, which would make modern, object-orientated development almost impossible.

Lastly, we define our error logs to a dedicated folder inside our projects so they can be found easily. You should make sure the error logs get filled correctly since some Apache installations make distinct assumptions about their location.

Finishing the script

In the last steps, we again make use of some of our variables to determine whether the user wants the pages to be activated automatically and whether we should write hosts-entries. In general, there is no reason not to do this. The only thing some users might be concerned about is whether we should write an SSL entry for the page, too.

read_page_name
check_project_existing
is_user_root
create_file_structure
create_vhost_file

if ${SSL_ACTIVE}; then
    create_vhost_ssl_file
fi

if ${AUTO_ENABLE_PAGE}; then
    a2ensite ${PAGENAME}.conf
    if ${SSL_ACTIVE}; then
        a2ensite ${PAGENAME}-ssl.conf
    fi
fi

if ${CREATE_LOCAL_HOSTS}; then
    sed -i "1s/^/127.0.0.1      $PAGENAME\n/" /etc/hosts
fi

systemctl restart apache2

In the beginning, we can find all our functions up to now. The reason for this is that we are now writing what could be referred to as the main function of the script, although it is not really a function, and the keyword main is nowhere to be found.

After the basic calls, we check whether the user wants SSL to be activated. Again, you don't really need this for development, so I chose to put it inside an if-statement. As a side note, here you can see the simplified versions of the if-statements, without the brackets and flag checking. Just plain, basic true-false checking. The SSL version of the Vhost file is slightly longer due to the fact it needs to know about some specific information like the certificate files.

create_vhost_ssl_file(){

    touch /etc/apache2/sites-available/${PAGENAME}-ssl.conf
    echo "<IfModule mod_ssl.c>
    <VirtualHost _default_:443>

        ServerAdmin $SERVERADMIN_EMAIL

        ServerName $PAGENAME
        ServerAlias www.$PAGENAME

        DocumentRoot $DIR$PAGENAME/public

        <Directory  $DIR$PAGENAME/public>
            Options Indexes FollowSymlinks
            AllowOverride All
            Require all granted
        </Directory>


        ErrorLog $DIR$PAGENAME/log/error.log
        CustomLog $DIR$PAGENAME/log/access.log combined

        SSLEngine on

        SSLCertificateFile $SSLCertificateFile
        SSLCertificateKeyFile $SSLCertificateKeyFile

        <FilesMatch \"\.(cgi|shtml|phtml|php)$\">
                        SSLOptions +StdEnvVars
        </FilesMatch>
        <Directory /usr/lib/cgi-bin>
                        SSLOptions +StdEnvVars
        </Directory>

    </VirtualHost>
</IfModule>" >> /etc/apache2/sites-available/${PAGENAME}-ssl.conf
}

Next, we enable the page(s). This checks the Vhost files and symlinks the entries into the sites-enabled folder in /etc/apache2/sites-enabled/. However, for the page to be callable, we need to restart the apache2 service. We should also write hosts-entries in our /etc/hosts file. This makes it possible to just type my-project.com or whatever you called it in your browser, instead of 127.0.0.1/whatever or localhost. Because we set up Vhost files with Servername entries, the calls will automatically be routed to the corresponding project folders.

Why the cryptic sed call?
The standard Ubunutu hosts file assumes that we list IPV6 entries beyond IPV4 entries. Because I ran into some minor problems when messing up this ordering, I decided to write a workaround. What this sed call does is basically writing the new entries at the very beginning of the file, pushing all entries one place further down. Don't think to hard into this, consider it black Unix magic and move on.

After that, we just restart the Apache server and should be able to call our test environment immediately.

Full code and video

I'd like to close this article out by posting the full code and a quick video of a sample use case. If you want to make the script into a shell command, you might want to put it into /usr/local/bin/. This way you can call it from everywhere. This is the preferred way of "installing" scripts locally, you should refrain from putting it into /bin since this place is usually meant for system-wide programs.

#!/bin/bash

DIR="/var/www/"
SERVERUSER="www-data"
SERVERADMIN_EMAIL="top@kek.mate"
SSL_ACTIVE=true
SSLCertificateFile="/etc/ssl/certs/ssl-cert-snakeoil.pem"
SSLCertificateKeyFile="/etc/ssl/private/ssl-cert-snakeoil.key"
AUTO_ENABLE_PAGE=true
CREATE_LOCAL_HOSTS=true

read_page_name(){

    echo -n "Name for new Page (without www): "
    read PAGENAME
}

is_user_root(){

    if [ $(id -u) -ne 0 ]
    then 
        echo "Please run the script as root!"
        exit 1
    fi
}

check_project_existing(){

    if [ -d "$DIR$PAGENAME/" ]; then
        echo "Error: Project ${PAGENAME} already existing!"
        exit 2
    else
        echo "Installing config files in ${DIR}${PAGENAME}/ ..."
    fi
}

create_sample_page(){

    touch ${DIR}${PAGENAME}/public/index.php
    echo "<html>
    <head>
        <title>$PAGENAME</title>
    </head>
    <body>
        <h1>Success! The $PAGENAME virtual host is working!</h1>
    </body>
</html>" >> ${DIR}${PAGENAME}/public/index.html
}

create_vhost_file(){

    touch /etc/apache2/sites-available/${PAGENAME}.conf
    echo "<VirtualHost *:80>

    ServerAdmin $SERVERADMIN_EMAIL

    ServerName $PAGENAME
    ServerAlias www.$PAGENAME

    DocumentRoot $DIR$PAGENAME/public

    <Directory  $DIR$PAGENAME/public>
        Options Indexes FollowSymlinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog $DIR$PAGENAME/log/error.log
    CustomLog $DIR$PAGENAME/log/access.log combined

</VirtualHost>" >> /etc/apache2/sites-available/${PAGENAME}.conf
}

create_vhost_ssl_file(){

    touch /etc/apache2/sites-available/${PAGENAME}-ssl.conf
    echo "<IfModule mod_ssl.c>
    <VirtualHost _default_:443>

        ServerAdmin $SERVERADMIN_EMAIL

        ServerName $PAGENAME
        ServerAlias www.$PAGENAME

        DocumentRoot $DIR$PAGENAME/public

        <Directory  $DIR$PAGENAME/public>
            Options Indexes FollowSymlinks
            AllowOverride All
            Require all granted
        </Directory>


        ErrorLog $DIR$PAGENAME/log/error.log
        CustomLog $DIR$PAGENAME/log/access.log combined

        SSLEngine on

        SSLCertificateFile $SSLCertificateFile
        SSLCertificateKeyFile $SSLCertificateKeyFile

        <FilesMatch \"\.(cgi|shtml|phtml|php)$\">
                        SSLOptions +StdEnvVars
        </FilesMatch>
        <Directory /usr/lib/cgi-bin>
                        SSLOptions +StdEnvVars
        </Directory>

    </VirtualHost>
</IfModule>" >> /etc/apache2/sites-available/${PAGENAME}-ssl.conf
}
test
create_file_structure(){

    mkdir ${DIR}${PAGENAME}
    mkdir ${DIR}${PAGENAME}/public
    mkdir ${DIR}${PAGENAME}/log

    create_sample_page

    chown -R ${SERVERUSER}:${SERVERUSER} ${DIR}${PAGENAME}
    chmod -R 755 ${DIR}${PAGENAME}
    chmod -R 775 ${DIR}${PAGENAME}log

}

############################################################################

read_page_name
check_project_existing
is_user_root
create_file_structure
create_vhost_file

if ${SSL_ACTIVE}; then
    create_vhost_ssl_file
fi

if ${AUTO_ENABLE_PAGE}; then
    a2ensite ${PAGENAME}.conf
    if ${SSL_ACTIVE}; then
        a2ensite ${PAGENAME}-ssl.conf
    fi
fi

if ${CREATE_LOCAL_HOSTS}; then
    sed -i "1s/^/127.0.0.1      $PAGENAME\n/" /etc/hosts
fi

systemctl restart apache2

And here is the script in action:

https://youtu.be/yIYTVTqk9Fs

Thanks for reading my article, if you have any further questions feel free to contact me.

Linuxapacheshell scriptDevopsvhosts