Category: Roles

Continuous Deployment with Codeship and WP Engine

Overview of continuous deployment pipeline

The purpose of this article is to share our approach to continuous deployment for WordPress plugins and themes on our servers at WP Engine. Continuous deployment is the automated testing and delivery of code to development and production environments. Knowing your web host’s requirements is essential to setting up a continuous delivery process. Once you know this, then you can evaluate a service and later develop a solution that meets your needs. I have implemented this for more than a dozen WordPress themes and plugins, and I believe that effort was worth the time it is saving us now.

Video Tutorial

Our Approach

Codeship automates the process of testing, building, and deploying code to your server, and was the first such product I found which would integrate with our web host (WP Engine). It watches a repository, clones an updated branch to a virtual machine, runs your commands on it, and then deploys the result to your servers. Services like Codeship allow us to automate our existing development and deployment workflow so that changes to the code in our Git repositories are automatically made on our servers.

We use three custom bash scripts in our Codeship projects: setup, build, and deploy. The setup script runs first and is mostly identical for all branches:

# Setup
# Get around restriction on shallow pushes by WP Engine
git filter-branch -- --all
git checkout branchname
# Add User Data
git config --global user.name "codeship-shortreponame"
git config --global user.email "youremail@email.com"
# Combine remote git servers
git remote add servers $SERVERNAME_ENVIRONMENT
git remote set-url --add --push servers $SERVERNAME_ENVIRONMENT
git remote set-url --add --push servers $SERVERNAME2_ENVIRONMENT
# Install needed modules
npm install -g grunt-cli
npm install -g grunt@0.4.0
npm install -g bower
npm install -g ruby
gem install compass
# Move repo files to a named folder
mkdir $FOLDERNAME
shopt -s extglob
mv !($FOLDERNAME) $FOLDERNAME
# Move repo files whose name begins with a period
mv .sass-lint.yml $FOLDERNAME/.sass-lint.yml
# Exclude development-only files from commit
rm .gitignore
mv .codeshipignore $FOLDERNAME/.gitignore
# Move named folder into a structure identical to the root directory of a WordPress server
mkdir -p $DIRECTORY
mv $FOLDERNAME $DIRECTORY
cd $DIRECTORY/$FOLDERNAME/

It adds git user data, defines a list of remote server repositories, and installs command line modules on the virtual machine. The staging servers are listed as remote git repositories in the staging branch’s setup script, and our production servers are listed as remotes in the master branch’s script. Then it replaces the repository’s gitignore file with our .codeshipignore file and moves all files into a folder structure relative to the root directory of our servers:

# RECOMMENDED BY WPENGINE
*~
.DS_Store
.svn
.cvs
*.bak
*.swp
Thumbs.db
# large/disallowed file types
# a CDN should be used for these
*.hqx
*.bin
*.exe
*.dll
*.deb
*.dmg
*.iso
*.img
*.msi
*.msp
*.msm
*.mid
*.midi
*.kar
*.mp3
*.ogg
*.m4a
*.ra
*.3gpp
*.3gp
*.mp4
*.mpeg
*.mpg
*.mov
*.webm
*.flv
*.m4v
*.mng
*.asx
*.asf
*.wmv
*.avi
# Directories that may or may not exist in repo AND should not be on the server
package.json
bower.json
.bower.json
.bowerrc
config.rb
node_modules
.sass-cache
*.md
*.txt
*.ai
*.scss
*.coffee
.gitignore
LICENSE
LICENSE-MIT
gruntfile.js
Gruntfile.js
werker.yml
.editorconfig
css/src
js/src
bower_components/**/foundation/scss/
bower_components/**/jquery/src/
bower_components/**/modernizr/*/
bower_components/**/modernizr/grunt.js
bower_components/**/picturefill/index.html
bower_components/**/picturefill/logos/*.png
bower_components/**/superfish/examples/
# Only using one file from bower_components/html5shiv
bower_components/html5shiv/
!bower_components/html5shiv/dist/html5shiv.js

Our typical build script imports Node and Bower modules and uses Grunt to compile Sass and Coffee files:

# Build
composer install
npm install
bower install
grunt develop

It uses the repo’s gruntfile (example) to perform a different set of tasks for the staging and production servers:

module.exports = (grunt) ->
  @initConfig
    pkg: @file.readJSON('package.json')
    watch:
      files: [
        'css/src/*.scss'
      ]
      tasks: ['sasslint', 'compass:dev']
    compass:
      pkg:
        options:
          config: 'config.rb'
          force: true
      dev:
        options:
          config: 'config.rb'
          force: true
          outputStyle: 'expanded'
          sourcemap: true
          noLineComments: true
    jsvalidate:
      options:
        globals:
          jQuery: true
          console: true
          module: true
          document: true
      targetName:
        files:
          src: [
            'js/*.js',
            'bower_components/foundation/js/vendor/fastclick.js',
            'bower_components/foundation/js/foundation/foundation?(.topbar).js',
            'bower_components/modernizr/modernizr.js',
            'bower_components/jquery/{dist,sizzle}/**/*.js',
            'bower_components/jquery-placeholder/*.js',
            'bower_components/jquery.cookie/jquery.cookie.js',
            'bower_components/respond/{cross-domain,dest}/*.js',
            'bower_components/html5shiv/dist/html5shiv.js'
          ]
    sasslint:
      options:
        configFile: '.sass-lint.yml'
      target: ['css/src/**/*.s+(a|c)ss']
    compress:
      main:
        options:
          archive: 'AgriFlex3.zip'
        files: [
          {src: ['AgriFlex/*.php']},
          {src: ['css/*.css']},
          {src: ['img/**']},
          {src: ['js/*.js']},
          {src: ['bower_components/fastclick/lib/fastclick.js']},
          {src: ['bower_components/foundation/{css,js}/**']},
          {src: ['bower_components/modernizr/modernizr.js']},
          {src: ['bower_components/jquery/{dist,sizzle}/**/*.js']},
          {src: ['bower_components/jquery-placeholder/*.js']},
          {src: ['bower_components/jquery.cookie/jquery.cookie.js']},
          {src: ['bower_components/respond/{cross-domain,dest}/*.js']},
          {src: ['bower_components/html5shiv/dist/html5shiv.js']},
          {src: ['vendor/**', '!vendor/composer/autoload_static.php']},
          {src: ['functions.php']},
          {src: ['README.md']},
          {src: ['rtl.css']},
          {src: ['screenshot.png']},
          {src: ['search.php']},
          {src: ['style.css']}
        ]

  @loadNpmTasks 'grunt-contrib-compass'
  @loadNpmTasks 'grunt-jsvalidate'
  @loadNpmTasks 'grunt-contrib-watch'
  @loadNpmTasks 'grunt-sass-lint'

  @registerTask 'default', ['compass:pkg']
  @registerTask 'develop', ['sasslint', 'compass:dev', 'jsvalidate']
  @registerTask 'package', ['compass:pkg', 'jsvalidate']

  @event.on 'watch', (action, filepath) =>
    @log.writeln('#{filepath} has #{action}')

Note that we use Compass’s expanded output with sourcemaps for developing Sass files and a compressed output for production. A web browser’s inspection tool can detect sourcemaps and use them to show you where a CSS rule is located in your Sass files. This is essential for troubleshooting or modifying Sass as quickly as possible.

Once the build script has finished, the deploy script commits changes to the cloned repository and pushes them to our WP Engine servers:

# Deploy
git add --all :/
git commit -m "DEPLOYMENT"
git push servers HEAD:master --force

If this is the first time you’re using the Git Push feature for a particular WP Engine install, you might need to use this git push command instead to avoid a “missing branch” error:

git push servers HEAD:refs/heads/master --force

We can only send code to WP Engine servers from Codeship using their git push feature. It replaces user input authorization with an SSH key. The .codeshipignore file allows us to exclude unneeded files from the commit and reduce upload times. Since our projects are authorized git push users with the WP Engine servers we push to, the last command pushes our final commit to the remote server repositories we defined during setup.

The video tutorial demonstrates how to implement this process. What follows are the same steps I take to connect each of our repositories to our WP Engine servers using Codeship.

Using Codeship to deploy Github repos to WP Engine

  1. Create a new Codeship project and connect it to a Git repo URL
  2. On the Codeship General Project Settings page, copy the SSH public key
  3. On each WP Engine install you want connected, use the SSH public key to add a new Git Push developer named codeship-shortreponame.
  4. Show what the variables page looks like On the Codeship project’s Environment settings page, create a new variable named SERVERNAME_PRODUCTION and SERVERNAME_STAGING for each WP Engine install you want connected.
  5. On the same page, define a FOLDERNAME variable as the name of the folder your repo’s files should be placed within. Also define a DIRECTORY variable as either wp-content/plugins or wp-content/themes.
  6. Show what the deployment pipeline page looks likeOn the Codeship project’s Deployment settings page, create one pipeline for the repo’s master branch and another for the staging branch.
  7. Create separate custom scripts in each pipeline for the setup, build (optional), and deploy portions of the process.
    • The setup script only has a few variables that need to be changed for each project: branchname, shortreponame, youremail@email.com, and SERVERNAME_ENVIRONMENT.
    • The build script contains your existing build process, such as importing NPM modules or preprocessing css and js files.
    • The deploy script has no variables.
  8. Ensure your Git repo has a master and staging branch.
  9. Checkout the staging branch of your repo and add a .codeshipignore file to its root directory using this template as a guide.
  10. Show what the build output page looks likeCommit the change and push it to your repository. This will trigger the Codeship build process, which you can watch on the project’s main page as it moves through each line of your custom scripts.
  11. Once the build has finished, it will send the code to the staging server if successful or stop on the first line that results in an error. If it is successful, it will be pushed to the server! Whether or not it is successful, you can click on any line of the script to see its output, so if it fails you can use this to troubleshoot that command.
  12. When your build is successful, you can merge the staging branch to the master branch on your computer and then push it to the repository. This will trigger the master pipeline, which will either deploy it to your production server or result in an error.

And that’s it! Now you have an automated deployment process integrated with your existing Git repository, and all of your servers can have the latest, tested version of your plugins and themes each time you push to it. If you found this article helpful, please leave a comment. You can view the public repositories of Agrilife Communications on Github for more information about their build processes. I can be reached by email at zachary.watkins@ag.tamu.edu.

Resources

Accessibility Resources from AccessU Conference

The Texas A&M IT accessibility services team attended the John Slatin AccessU conference hosted by Knowbility in Austin, Texas on May 17-18, 2017. From the various sessions, the team learned about new accessibility resources you can incorporate into your design, development and work processes.

If you have experience with any of these, please write a comment below. We would love to hear if these are beneficial.

Addressing the struggles of making sites accessible and usable

Accessibility and Usability Vin Diagram

Credit: Unknown

 

A question was asked at an Accessibility Special Interest Group meeting along the lines of, “Do we have access to a report that shows all the accessibility issues for our sites?” A comment was then made, “I just don’t know where to start.” I immediately reflected back to when Erick Beck and I gave a presentation along with representatives from SiteImprove of the various components of the SiteImprove suite. The same type of question was asked during the Q&A section. I felt it was time to write an article to share with the entire web community on campus my thoughts on bringing every site into compliance making our sites usable.

Triage known issues

The first step I recommend is triaging known issues on all your sites collectively and establishing a plan. Start by looking at reports, whether in SiteImprove or another reporting tool, and find similarities or patterns. For example, if you notice a contrast ratio issue on all your reports and the same theme is used on each site, then a simple CSS fix to the theme and your theme’s source files will resolve the issue.

Identify teaching opportunities

If you see a lot of issues on alternative text, such as filenames used as alternative text, use it as a teaching moment. Send a polite email to your content contributors stating that you have noticed a lot of images added with the filename used as the alternative text. Provide simple steps to correct the issue and avoid mentioning the rules or policies associated with it. Please do not misunderstand me and think that I do not believe the accessibility rules and policies are not important. The rules and policies are important but, the idea here is to gingerly tell those who contribute content to the site to be mindful of certain things. We don’t want to beat them into the ground with the rule book.

Make the content usable

Instead of embedding an image containing paragraphs of text, tables and supporting images, make the image accessible by making it usable. We all agree that we want Google, Bing and any other search engine to index our site. Why should we then mask our content within an image? We want visitors to be able to find something on a page. Why should we put all our text in an object that can’t be searched? We want those looking at our site with a mobile device to read what we have to say without pinching to zoom in and out. Why include it in an image that scales automatically making the text difficult to see? Extract the content from the image and place it on the page so that everyone can read it making is usable and accessible.

Remember, the end goal is to have our sites accessible by making them usable. Try your best not to feel overwhelmed or frustrated.

  • Start by triaging the reported issues and establish a plan.
  • Identify the issues that pertain to a theme or template where one fix resolves the issue in several places.
  • Use your analytics to identify your top 25 or 50 most visited pages and resolve all the issues on those pages.
  • Work your way down to the fourth, fifth and sixth level pages eliminating the issues reported by your scanning tool.
  • Finally, you can review your site to check the tabbing order or if the appropriate landmarks are being used as both of these items need human interaction to verify they are within the guidelines.

Help is on the way!

Don’t be afraid to ask questions! The odds are high that someone has been in the same circumstance and can help you along the way. Join the accessibility sprint on July 14th where the goal is to bring people and technology together to improve the usability of our sites. This is a great opportunity to interact with others across campus who are also making updates that impact accessibility!

Web Technologies & Development at 2017 Tech Summit

2017 Texas A&M University System Technology Summit
Moody Gardens Convention Center, Galveston, TX
February 20-22

The Texas A&M University System Technology Summit is the place technology experts come to learn from the best, exchange ideas on common challenges and spend time together. Professionals have the rare chance to blend technical learning across a wide range of subjects. Tech Summit offers sessions to help you master your daily work, while enjoying a taste of island time. This year’s web track is full of valuable sessions:

Pre-conference workshops specific to web development are:

GoWeb After Dark

Equidox by Onix treats you to hors d’oeuvres and a complimentary first drink. Join the Texas A&M GoWeb group and end your day in a casual setting. Network with web professionals and discuss everything web, from the latest trends on analytics, accessibility and branding to recent successes and challenges.

Please check out the complete schedule and we hope to see you in Galveston!


Donald St. Martin
Track Chair, Web Technologies & Development

HTTPS and the CAS Maestro WordPress plugin

At the College of Engineering, we are in the process of adding SSL certificates to all of our WordPress sites. On top of faster load times with HTTP/2 and higher page rankings in Google’s search engine, we also wanted our WordPress logins to be secure. It is a simple procedure that adds one more layer of obfuscation. Well, it was supposed to be a simple procedure but I ran into an issue when securing the WordPress login process.

Continuous Release on Github with Grunt

TL:DR; Just show me the Gruntfile!

This post will describe the release tasks I’ve written for Grunt which work seamlessly with our existing continuous deployment pipeline on Codeship. I also mention a step you can take if you want to do this directly from your command line instead.

I help develop WordPress themes and plugins for Texas A&M AgriLife, which are available on Github. Earlier this year, I was asked to develop and implement a continuous deployment pipeline for most of our repositories. I wrote an article about it here. Long story short, we use Codeship to watch our repository branches (staging and master) and run console commands to prep and push them to the staging and production versions of our servers.

I was recently asked to create a release for our latest theme AgriFlex3, and in hope I looked for articles on how to do so with Grunt. I soon found this article by Paul C. Pederson which showed me this approach was possible, but I needed something a little different. What follows are Grunt modules and commands based on those I use to package and release our open-source projects on Github with a release message that describes all of the commits made since the previous release. These tasks can be run for any project, not just WordPress!


grunt-contrib-compress

This is the simplest task. You provide the archive file name and an array of file globs to copy into it. We try to use values from package.json when we can. View the grunt-contrib-compress repository for more information.

compress:
  main:
    options:
      archive: '<%= pkg.name %>.zip'
    files: [
      {src: ['css/*.css']},
      {src: ['img/**']},
      {src: ['js/*.js']},
      {src: ['**/*.php']},
      {src: ['*.php']},
      {src: ['README.md']},
      {src: ['screenshot.png']}
    ]

grunt-gh-release

This task creates our release using properties within package.json, a Github access token available in our Codeship environment, and a few custom tasks (shown in the next section) that populate the release message beforehand. If you want to run your release task locally then use the documentation’s method for the options token instead of what is shown here. View the grunt-gh-release repository for more information.

gh_release:
  options:
    token: process.env.GITHUB_PERSONAL_ACCESS_TOKEN
    owner: 'agrilife'
    repo: '<%= pkg.name %>'
  release:
    tag_name: '<%= pkg.version %>'
    target_commitish: 'master'
    name: 'Release'
    body: 'First release'
    draft: false
    prerelease: false
    asset:
      name: '<%= pkg.name %>.zip'
      file: '<%= pkg.name %>.zip'
      'Content-Type': 'application/zip'

Custom tasks

The two custom tasks I’ve written will run console commands and handle their output. The first task gets the last release tag and uses it to set a config value as the range of commit messages we want to retrieve. The second task takes this range and uses it to populate the release message with a bulleted list of commits below their author’s name. This is necessary because the github token is linked to a person and that name shows near release messages. We want to ensure they are not seen by users as the author of all released commits.

@registerTask 'setreleasemsg', 'Set release message as range of commits', ->
  done = @async()
  grunt.util.spawn {
    cmd: 'git'
    args: [ 'tag' ]
  }, (err, result, code) ->
    if(result.stdout!='')
      # Get last tag in the results
      matches = result.stdout.match(/([^\n]+)$/)
      # Set commit message timeline
      releaserange = matches[1] + '..HEAD'
      grunt.config.set 'releaserange', releaserange
      # Run the next task
      grunt.task.run('shortlog');
    done(err)
    return
  return
@registerTask 'shortlog', 'Set gh_release body with commit messages since last release', ->
  done = @async()
  grunt.util.spawn {
    cmd: 'git'
    args: ['shortlog', grunt.config.get('releaserange'), '--no-merges']
  }, (err, result, code) ->
    if(result.stdout != '')
      # Hyphenate commit messages
      message = result.stdout.replace(/(\n)\s\s+/g, '$1- ')
      # Set release message
      grunt.config 'gh_release.release.body', message
    else
      # Just in case merges are the only commit
      grunt.config 'gh_release.release.body', 'release'
    done(err)
    return
  return

Gruntfile.coffee

module.exports = (grunt) ->
  @initConfig
    pkg: @file.readJSON('package.json')
    compress:
      main:
        options:
          archive: '<%= pkg.name %>.zip'
        files: [
          {src: ['css/*.css']},
          {src: ['img/**']},
          {src: ['js/*.js']},
          {src: ['**/*.php']},
          {src: ['*.php']},
          {src: ['README.md']},
          {src: ['screenshot.png']}
        ]
    gh_release:
      options:
        token: process.env.GITHUB_PERSONAL_ACCESS_TOKEN
        owner: 'agrilife'
        repo: '<%= pkg.name %>'
      release:
        tag_name: '<%= pkg.version %>'
        target_commitish: 'master'
        name: 'Release'
        body: 'First release'
        draft: false
        prerelease: false
        asset:
          name: '<%= pkg.name %>.zip'
          file: '<%= pkg.name %>.zip'
          'Content-Type': 'application/zip'

  @loadNpmTasks 'grunt-contrib-compress'
  @loadNpmTasks 'grunt-gh-release'

  @registerTask 'release', ['compress', 'setreleasemsg', 'gh_release']
  @registerTask 'setreleasemsg', 'Set release message as range of commits', ->
    done = @async()
    grunt.util.spawn {
      cmd: 'git'
      args: [ 'tag' ]
    }, (err, result, code) ->
      if(result.stdout!='')
        # Get last tag in the results
        matches = result.stdout.match(/([^\n]+)$/)
        # Set commit message timeline
        releaserange = matches[1] + '..HEAD'
        grunt.config.set 'releaserange', releaserange
        # Run the next task
        grunt.task.run('shortlog');
      done(err)
      return
    return
  @registerTask 'shortlog', 'Set gh_release body with commit messages since last release', ->
    done = @async()
    grunt.util.spawn {
      cmd: 'git'
      args: ['shortlog', grunt.config.get('releaserange'), '--no-merges']
    }, (err, result, code) ->
      if(result.stdout != '')
        # Hyphenate commit messages
        message = result.stdout.replace(/(\n)\s\s+/g, '$1- ')
        # Set release message
        grunt.config 'gh_release.release.body', message
      else
        # Just in case merges are the only commit
        grunt.config 'gh_release.release.body', 'release'
      done(err)
      return
    return

 

If this helps you, or if you have any suggestions for improvements, please leave a comment!

Tools discussed at the GoMobile retreat

  • Foundation 5 – Latest mobile-first, responsive front end framework from Zurb
  • Codepen.io / Jsfiddle? – Allow real-time testing of html/css/js snippets
  • Regexr – Real time testing of regular expressions and great regex documentation
  • Browserstack – Test web sites on live virtual machines with just about every web browser.? Many mobile devices and OSs available as well.
  • Siteimprove – Web governance, scans for broken links, spelling and grammar problems, unwanted content, accessibility problems and more.
  • Ghostlab – Synchronized web development and testing
  • BrowserSync.io – Open source synchronized browser testing
  • Ghost Inspector – Automate front-end testing, checks the operation of specific functionality and page elements
  • Invision – Tool for collaboration on prototypes and mockups
  • Macaw – Responsive prototyping tool, outputs?HTML5 and css (also Froont, WebFlow, Adobe Muse, PineGrow)
  • Google Web Designer – Creates interactive HTML5-based designs and motion graphics, good for custom banners (Also Adobe Edge Animate)
  • Microsoft Edge Browser / Firefox Developer Edition

EDUCAUSE 2016

October 25-28, 2016

Anneheim, CA

www.educause.edu

The EDUCAUSE Annual Conference is the premier convening of IT professionals and technology providers across the diverse higher education landscape. The conference creates networking opportunities for colleagues to share ideas regarding strategies, leading change, effective processes, what’s working, and sometimes—more importantly—what isn’t.

HighEdWeb 2016 Annual Conference

October 16-19, 2016

Memphis, TN

2016.highedweb.org

HighEdWeb is the annual conference of the Higher Education Web Professionals Association, created by and for all higher education Web professionals—from programmers to marketers to designers to all team members in-between—who want to explore the unique Web issues facing colleges and universities.