Database Theory, Elasticsearch, Search, WordCamps, WordPress Plugins

ElasticPress at WordCamp Paris

This weekend I presented at WordCamp Paris 2015. My session was titled “Modernizing WordPress Search with Elasticsearch”. The talk ran through issues with WordPress search, what Elasticsearch is, setting up an Elasticsearch cluster, and configuring ElasticPress.

Elasticsearch is a very exciting technology and I am thrilled at the chance to spread information about it. I (and 10up in general) am very proud of the work we have done on ElasticPress. My hope is that more people will install the plugin and give us feedback as a result of the talk.

Here are my slides for the talk:

Don’t forget that comprehensive documentation for ElasticPress lives on Github.

Publishing Experience, WordPress Plugins, WP API

Custom Contact Forms Redux

In 2009-2010 Custom Contact Forms was one of the first WordPress plugins I wrote. The purpose of the plugin was to solve a simple problem: easy contact form building on the web.

As I was building the plugin, WordPress 3.0 had not yet been released, and therefore custom post types and other useful API’s did not yet exist. I wrote the plugin the best I could using custom database tables (ouch!). The plugin become decently popular. I continued development on the plugin for the next year or so. In the process I learned a lot about writing code for WordPress. Also in the process I landed a job at 10up where I learned (and continue to learn) more than I ever thought I would.

Years later in 2015, here I am an experienced WordPress developer and an avid open source contributor. I have used most of the popular WordPress form plugins: Gravity Forms, Ninja Forms, Formidable, etc. Using all these plugins, despite lots of great functionality, have left me with the nagging feeling that something is missing.

That something was the WordPress experience. With the release of the media manager in WordPress 3.5, we have all grown accustomed to a high quality media management experience through smooth JavaScript interactions. Form management should be no different; they should be built within a media manager-esque modal, no page reloads should be necessary, and we should get live previews of what we are building. Simple task right? 8 months later… :)

Detailed installation and usage instructions are on Github. Download the plugin from

For Users

Within the post edit screen, a simple “Add Form” button next to the “Add Media” button brings up the form manager modal:


Within the form management modal, you can also see your existing forms and edit them if you choose:

View all your forms within the form management modal for easy access.

After inserting a form into a post, you see a nice preview within TinyMCE:

Form previews even show within TinyMCE.

While the meat of the plugin lies within the form manager, single form views exist. Within a single form view you can see a live form preview:

Live previews of your forms are generated on the fly. No more guess and test.

Also within the single form view lives the form submissions table. You can easily paginate through results and add/remove columns as you please:

Easily scroll through form submissions in tabular format. You can configure columns you would like to see to ensure an uncluttered view.

For Developers

Most of the plugin is written in JavaScript. Custom Backbone views, models, and collections are written to emulate and display forms and fields. The plugin is extremely extensible. You can easily hook in to modify existing fields and views as well as create your own.

The plugin includes the new JSON REST API for WordPress. Right now, it is included as a Composer dependency for various reasons until the API is added to WordPress core.

Note: While the plugin is suitable for production environments, version 6 is still somewhat in beta. Please let me know on Github if you experience any problems.

Elasticsearch, Search, WordPress Plugins

Valuable Lessons Learned in ElasticPress

ElasticPress is a 10up WordPress plugin project that integrates Elasticsearch with WordPress. As we all know search in WordPress is not a great experience. Why? Well, MySQL is not a database optimized for search. Thus ElasticPress was born.

1. Search result relevancy scores on sites with high post to shard ratios can vary depending on order of indexing.

We first noticed this in our integration testing suite. We were using three shards across 1 primary node. Depending on the order that posts were indexed, different relevancy scores were returned for the same search.

Elasticsearch relevancy scores are calculated as term frequency / inverse document frequency. Term frequency is the number of times a term appears in the query field of the current document (or post). Inverse document frequency measures how often the term appears in all query fields across all documents in the index of the current shard. Notice I said shard NOT index. The shard a post lives on is determined by the number of shards and the size of the index. We can’t exactly predict relevancy scores for a search on an index across more than one shard. The Elasticsearch documentation has a great article on this.

The solution for testing purposes is to only use one shard. In the real world, this shouldn’t matter as inconsistencies plateau as index sizes grow larger. However, this is still something to be aware of.

2. There is no right search algorithm for WordPress. Fine tuning algorithms is an on-going, collaborative process.

As of ElasticPress 1.1, the meat of our default search query looked like this:

  "query": {
    "bool": {
      "must": {
        "fuzzy_like_this": {
          "fields": [
          "like_text": "search phrase",
          "min_similarity": 0.75

fuzzy_like_this is great. It combines fuzzy and more_like_this queries. fuzzy searches against a set of fuzzified terms (using the levenshtein distance algorithm). more_like_this selects “interesting” terms based on a number of factors like document frequency and checks each document against those terms.

The problem we encountered was that in certain established indexes exact matches were not getting boosted to the very top of results. This was due to the way the fuzzy_like_this algorithm works. We added an extra query to our search algorithm in 1.2 to boost exact matches:

  "query": {
    "bool": {
      "should": [
          "multi_match": {
            "query": "search phrase",
            "boost": 2,
            "fields": ["post_title", "post_content", "post_excerpt"]
          "fuzzy_like_this": {
            "fields": ["post_title", "post_excerpt", "post_content"],
            "like_text": "search phrase",
            "min_similarity": 0.75

The should query tells Elasticsearch that one of the multi_match or fuzzy_like_this queries must be true for a document to match. It then boosts anything found multi_match x2.

This solved our immediate problem but is not the perfect algorithm. We expect to continually optimize this for WordPress over time. (Note that ElasticPress allows you to filter the search query entirely if you want to customize it.)

3. Disable indexing during imports.

By default ElasticPress indexes when a post is created. This is great until you try to import a few thousand posts, and your Elasticsearch instance gets overloaded. This bit us pretty hard. As of newer versions, ElasticPress disables syncing during WordPress imports big or small.

Granularly Manage Per-Post Access with Editorial Access Manager Plugin

Given the unmet need to allow users only access to edit specific pieces of content, I created a simple plugin called Editorial Access Manager.

By default in WordPress, we can create users and assign them to roles. Roles are automatically assigned certain capabilities. However, sometimes default roles are not enough, and we have one-off situations. Editorial Access Manager lets you set which users or roles have access to specific posts (as well as pages and custom post types).

Only the Administrator and Editor role have access to manage pages by default in WordPress. Making a user an Editor gives them a lot of power. I ran into the situation where we wanted a Contributor (less power than an Editor) to be able to edit only a few pages hence the need for this plugin.

My plan is to eventually integrate this plugin with EditFlow allowing people to assign User Groups to have access to specific pieces of content.

Any development help on this project would be much appreciated! Fork the Github repo.

Command Line WordPress, WordCamps, WordPress Plugins

WordCamp Ottawa

This weekend I was lucky enough to present at the 2nd annual WordCamp Ottawa. My talk was titled “Saving Time with WP-CLI”. WP-CLI is a plugin that lets you interact with your WordPress installations via the command line. My talk runs through some cool built-in WP-CLI commands as well as creating custom commands. My slides for the talk are here:

Pull Feed Content into WordPress

For the past year or so I’ve been using Automattic’s Syndication plugin to pull feed content into my WordPress plugin. Feeds that I designate are pulled in as posts using WordPress Cron. I decided while the plugin worked well for the most part, it wasn’t quite meeting my needs. This prompted me to build a new plugin called Feed Pull.

Here are some differences between the two plugins:

  • Feed Pull is a plugin for pulling content out of XML feeds. Syndication has feed pulling as well as content pushing features.
  • Feed Pull has a much friendlier feed management screen. Both Syndication and Feed Pull use the WordPress post edit, screen, however Feed Pull offers a much cleaner experience from the way options are presented to the instructions provided. At the moment Feed Pull does not have all the features of Syndication such as taxonomy and constant field mapping. However, Feed Pull probably has everything you need and is very extensible.
  • Feed Pull has far superior error logging to Syndication.
  • Feed Pull allows you to schedule content pulling in the future.
  • Feed Pull allows you to do manual pulls using AJAX rather than a sometimes frustrating one-time cron job.

Feed Pull contributions are always welcome on Github.

Introducing Post Customizer

10up just released a new plugin called the Post Customizer. I, along with John James Jacoby, John Bloch, Drew Jaynes, and Carl Danley led the charge on the plugin. The idea of the plugin is to mimic the theme customizer for posts. The plugin adds functionality that displays an overlay when you click “Preview” within the post editor. The overlay has a sidebar allowing you to edit the excerpt and featured post. In the middle of the overlay is a frame that shows the front of the website. Within that frame you can edit the post title and content. This enables you to make “live” changes and to see how they look as you make them. There are other plugins that offer similar behavior. However, Post Customizer closely follows the Theme Customizer and thus WordPress standards.

We want to make this plugin super extensible, and we need all the help we can get. We would appreciate contributions! Fork the plugin on Github.

Safe Redirect Manager

Hey Everyone!

I wanted to let everyone know about a new WordPress plugin called Safe Redirect Manager. I am one of the main developers of this plugin, and it has just been released on along with VIP (which is quite a special honor).

So what is the difference between this plugin and all the other redirect management plugins? Well, this plugin is simple, safe, and straight to the point. Instead of using the WordPress options table to store redirects, it uses a custom post type; this is much more flexible and lighter on your server. The plugin has been tested and reviewed by multiple WordPress core contributors and is available on — in order for a plugin to be available on it must be extremely secure. Safe Redirect Manager also allows you to use regular expressions in your redirects (if you don’t know what this is, don’t worry it’s an optional feature).

Download here:

P.S: If you are a developer, Safe Redirect Manager is available to be forked on Github.

Edit: Thank you Branco Radenovich for the Slovak translation. This will be included in version 1.6.1 of the plugin.