Synchronize Data With lftp Using GitLab CI/CD

For a lot of websites I build I use git for version control. I always used ftp client such as FileZilla to upload files to web hosting. But it gets annoying as you got more websites to manage. You have to login to ftp, click through GUI to find your folders. It is time consuming. So one day I decided to use GitLab CI/CD to build and deploy websites to web hosting.

Prepare yourself

Suppose we have already set up web hosting with ftp access. Now we need only create .gitlab-ci.yml in our repository and tell it to deploy. How do we do that? For most of my websites I use only static files so I don’t need the build step (example later). All I do is copy (it is more of a synchronizing) files using lftp to my web hosting. For this I use alpine:latest docker image with installed utility lftp.

You can see I set the image I want to use, then install the lftp utility using apk add lftp (similar to apt-get install). Then we do the deploy itself using

lftp -e "set ssl:verify-certificate no; open $FTP_HOST; user $FTP_USERNAME $FTP_PASSWORD; mirror -X .* -X .*/ --reverse --verbose dist/ www/; bye`

Let’s break down the command. First we call the program, obviously, with -e option which means execute following by our commands. We turn off ssl verification (if your hosting doesn’t support it). Then we open the connection to your ftp server using open <FTP_URL>, set user and password user <USERNAME> <PASSWORD>. Now comes the copying part. mirror has few options. These used tell the program to ignore . and .*/ as we don’t want to synchronize hidden files. Next we supply --reverse option which tells the lftp to upload local files to remote (lftp is set the opposite by default, meaning it would download from ftp to local). The --verbose option just logs out details about transfer and lastly we specify the directories dist and www on hosting. Now we only need to say bey to end the command.

One more option worth mentioning is --delete which also removes files from remoter server when they don’t exist on local.

If you need some other tools for building your project before deploy it just use different alpine image. For example for project built on node I use mhart/alpine-node docker image. The deploy part is the same. Example follows.

image: mhart/alpine-node

stages:
    - build
    - deploy

before_script:
    - apk add lftp

build:
    stage: build
    only:
        - master
    script:
        - npm ci
        - npm run build
    artifacts:
        paths:
            - dist

deploy:
    stage: deploy
    only:
        - master
    script:
        - lftp -e "set ssl:verify-certificate no; open $FTP_HOST; user $FTP_USERNAME $FTP_PASSWORD; mirror -X .* -X .*/ --reverse --verbose dist/ www/; bye"

And thats it. Happy deploying.

Warning! The `mirror` command is not always your best bet.

As pointed out in comments by Pavel Kutáč, the mirror command is not reliable with some FTP servers. See his post on the topic for more information.

Sources:

https://www.savjee.be/2019/04/gitlab-ci-deploy-to-ftp-with-lftp/

https://mrkaluzny.com/how-to-deploy-any-project-using-ftp-with-gitlab-continous-integration-ci/