# Adding shortcodes to the Marksmith editor
Occasionally, when creating content using an editor, be it Markdown or WYSIWYG, we need specific parts that exceed standard formatting options.
Whether it's highlighting important information, adding visually enriched snippets or embedding third-party content, the basic editor features often fall short.
This is where adding a short code or callout feature is useful.
In this article, we will learn **how to add shortcode support to the [Marksmith editor](https://avohq.io/blog/ruby-on-rails-markdown-editor-marksmith)** by building a blog with enriched content abilities.
Let's start by understanding what shortcodes are:
## What are shortcodes?
Shortcodes, a term popularized by the WordPress CMS, are a way to add enriched content using a text editor, without having to do any custom coding or even knowing about the underlying implementation.
They are also known as **callouts, custom content blocks, components, among other terms.**
Some use cases for them include:
- **Enriched existing HTML tags:** like introducing a testimonial using a `blockquote` tag that also includes a user picture and name, or a `button` that might trigger a newsletter subscription dialog.
- **Third party content embeds:** think adding a YouTube video or a podcast player that plays a specific episode, etc. Some editors have integrations with services like Embedly, but we can avoid using those with shortcodes.
- **Complex widgets:** this includes things like image galleries, interactive components, calculators, real-time data components like a sport results widget, etc.
The applications are numerous, and they actually depend on the business requirements.
{{info}}
This callout you're reading was actually created with a shortcode using the Marksmith editor. It receives the content as a parameter and then renders a styled template that results in what you're seeing right now.
{{/info}}
## What we will build
To demonstrate how the Marksmith editor works, we will build a simple blog with a `Post` model that has `title`, `excerpt`, `body` and `cover` attributes.
We will use a Rails scaffold to generate the views so we don't waste much time on that part which, in this case, is not that important.
Then, we will add support for shortcodes using two formats:
- **Braces syntax:** shortcodes are defined using double braces around the name of it and have a closing `/}}` syntax. They can be self-closing or have an opening and closing tag.
- **VitePress custom containers syntax:** the shortcodes will use a syntax like `::: info` to start the block and `:::` to mark the end of the block.
We will also have self-closing shortcodes `{{newsletter_signup /}}` and the support to add attributes to the shortcodes in case our templates need them.
The final result should look something like this:
{{gumlet video_id="67edc16bf7cb247edeedd140" width="1086" height="896" /}}
So, without further ado, let's start by setting our application up:
## Application setup
Let's start by creating the application:
```console
$ rails new marksmith_codes --css=tailwind --javascript=esbuild
```
Next, we will install ActiveStorage:
```console
$ bin/rails active_storage:install
```
Then, we will add a `Post` scaffold:
```console
$ bin/rails generate scaffold Post title excerpt body:text
```
We run the migrations to create the actual tables:
```console
$ bin/rails db:migrate
```
Now, let's add and configure Marksmith:
### Marksmith setup
The first step is to add the gem to our Gemfile and install it:
```console
$ bundle add marksmith && bundle install
```
Next, we have to add the `@avo-hq/marksmith` NPM package:
```console
$ yarn add @avo-hq/marksmith
```
Or, if you're using importmap:
```console
$ bin/importmap pin @avo-hq/marksmith
```
Next, we will import and register the required controllers. Note that the `ListContinuationController` is optional, but I think it's usually the expected behavior, so we will add it:
```javascript
// app/javascript/controllers/index.js
import { MarksmithController, ListContinuationController } from "@avo-hq/marksmith"
application.register("marksmith", MarksmithController)
application.register("list-continuation", ListContinuationController)
```
Then, we add the Marksmith styles to our layout:
```erb
<%# app/views/layouts/application.html.erb %>
<%= stylesheet_link_tag "marksmith" %>
```
To finish everything, we replace the `text_area` in the `_form` partial with the `marksmith` method of the custom Marksmith `FormBuilder` extension:
```erb
```
Finally, and after some styling tweaks using Tailwind, the `Post` form should look something like this:
Now, we can create a post and render the HTML using the `<%== marksmithed @post.body %>` helper, and it should look something like this:
By the way, the Marksmith editor is one of the text fields you can use with Avo, our [Rails admin](https://avohq.io/rails-admin) and internal tooling gem.
{{info}}
There are two ways to render the Marksmith editor: one is using the `<%= marksmith_tag :body %>` helper, and the other is using the form-specific helper `<%= form.marksmith :body %>`.
{{/info}}
## Adding the shortcodes feature
The core of adding shortcodes or callouts is parsing the content and searching for a given pattern that matches what we expect is an actual shortcode and replacing that with a template.
For example, if we find an `{{example}}Content{{/example}}` shortcode inside a Markdown field, we could replace that with the actual template HTML:
```html
Hello from the example shortcode template
```
This use case seems straight-forward enough: we just use a Regex to identify the shortcode and replace it with the actual HTML, but we actually need a way to make this dynamic and extensible.
If you've been paying attention, we used the `<%== marksmithed @post.body %>` helper to render our posts content and that's ok because the gem is actually using Redcarpet under the hood to render the HTML from our markdown content.
But, if we want to customize how our Markdown is rendered, we will need to create a custom renderer and for that, we need to use the `redcarpet` gem.
The first step is to add the gem and install it:
```console
$ bundle add redcarpet && bundle install
```
Now, we can start actually building the feature by adding a custom Markdown renderer:
### The custom renderer
The first step is to actually create a custom renderer under the `lib` folder that will parse the shortcodes to render the actual HTML.
By default, the library works by taking a Markdown parser that receives a renderer that, by-default, can output HTML or XHTML.
What we will do is create a `CustomRenderer` which inherits from `Redcarpet::Render::HTML` and receives Markdown text, parses it using the configuration we pass through the `extensions` method and returns HTML using the config we pass to it with the `renderer_options`:
```ruby
class CustomRenderer < Redcarpet::Render::HTML
def initialize
super(renderer_options)
@markdown = Redcarpet::Markdown.new(self, extensions)
end
def render(text)
@markdown.render(text)
end
private
def extensions
{
lax_spacing: true,
fenced_code_blocks: true,
tables: true,
space_after_headers: true,
autolink: true,
highlight: true,
quote: true,
strikethrough: true,
}
end
def renderer_options
{
hard_wrap: true,
link_attributes: { target: "_blank" }
}
end
end
```
If you'd like to find out about what every config option is doing, feel free to [check Redcarpet's documentation](https://github.com/vmg/redcarpet).
Now, if we render the content using this custom renderer:
```ruby
class Post < ApplicationRecord
## Rest of the code
def formatted_body
CustomRenderer.new.render(body)
end
end
```
```erb
<%= sanitize @post.formatted_body %>
```
We would get something very similar to what we saw above using the `marksmithed` helper.
### Custom renderer using kramdown
Besides Redcarpet, Marksmith allows us to use `kramdown` as a renderer, let's see how we can implement our custom renderer using it.
The first step is to add the gem to our Gemfile and make sure it's installed:
```shell
$ bundle add kramdown && bundle install
```
Now, let's create the `CustomRenderer` with the ability to render HTML starting from Markdown:
```ruby
# lib/custom_renderer.rb
class CustomRenderer
def render(markdown)
document = Kramdown::Document.new(markdown, parser_options)
document.to_html
end
def parser_options
{
auto_ids: true,
hard_wrap: true,
syntax_highlighter: 'rouge',
syntax_highlighter_opts: { css_class: 'highlight' },
link_attributes: { target: '_blank', rel: 'noopener noreferrer' }
}
end
end
```
As you can see, there's no need to inherit from any class like we did in the RedCarpet example, we just need to define a `render` method that accepts markdown text and we assign a `Kramdown::Document` instance to the `document`. The kramdown documents responds to `to_html`and produces the desired output.
Now, we can do exactly the same we did with the Redcarpet version:
```ruby
class Post < ApplicationRecord
## Rest of the code
def formatted_body
CustomRenderer.new.render(body)
end
end
```
```erb
<%= sanitize @post.formatted_body %>
```
Now, we should get the same result we got using Redcarpet.
Further down this article, we will be showing most of the examples with the Redcarpet renderer, but consider that `kramdown` also implements the `preprocess` method which makes everything compatible. We will make a note when a change needs to be made.
{{info}}
If you are keen to explore a third rendering option, we wrote an article about [GitHub flavored Markdown using the Commonmarker gem](https://avohq.io/blog/github-flavored-markdown-commonmarker), feel free to read it and tell us what you think.
{{/info}}
Now that we're at a similar starting point, let's start by adding our first shortcode:
### Adding self-closing shortcodes
These are shortcodes that don't need to “receive” content. They will appear whenever we enclose the shortcode name in double curly braces with a closing slash: ``{{newsletter /}}``
To add them, we will use the `preprocess` callback that Redcarpet provides, which receives the full Markdown document.
We just need to define the method and do our processing in there:
```ruby
class CustomRenderer < Redcarpet::Render::HTML
def initialize
super(renderer_options)
@markdown = Redcarpet::Markdown.new(self, extensions)
end
def preprocess(document)
## Here
end
def render(text)
@markdown.render(text)
end
## Rest of the code
end
```
Rendering will be a two-step process: matching the shortcode using a regular expression and replacing it with HTML generated in a partial.
So, to handle self-closing shortcodes we need the following:
```ruby
def preprocess(document)
document.gsub!(/\{\{(\w+)\s*\/\}\}/) do |match|
name = $1
render_shortcode(name)
end
document
end
```
What the Regex does is:
1. Matches two opening curly braces: ``{{``.
2. Captures one or more word characters (letters, numbers, underscores) and puts that into the first capture group. That's where we get the `name` from.
3. Matches zero or more white spaces, so `{{shortcode/}}` and `{{shortcode /}}`both work.
4. Matches a `/` followed by two closing curly braces: `}}`.
Then, we define the `render_shortcode` method:
```ruby
def render_shortcode(name)
result = ApplicationController.render(
partial: "shortcodes/#{name}",
layout: false,
)
result
rescue => e
Rails.logger.error("Error rendering shortcode #{name}: #{e.message}")
"{{#{name} /}}"
end
```
It tries to render a partial that has the shortcode's name passing `layout: false` so we get the partial itself.
If that fails for some reason, we log the error to the console and return the shortcode as it was defined originally.
You might want to return `nil` or an empty string to avoid rendering textual shortcodes. I actually added like that because I need to cite the shortcodes, but you might have different needs.
Now, we will try adding a `banner` shortcode to the content and add the partial inside `app/views/shortcodes` to see if everything is working:
Unfortunately, we're getting this, which doesn't seem to be what we want.
This error occurs because, since version 6.1, Rails includes the `action_view.annotate_rendered_view_with_filenames` config option that produces HTML comments around each partial to help us debug our applications.
The latest version of Rails, which is [version 8](https://avohq.io/blog/rails-8-whats-new), happens to have it by default, so we have to disable it around the shortcode rendering process to make it work correctly.
Before rendering the shortcode we store the config in a variable, we disable the feature, and then we return to the previous configuration:
```ruby
# lib/custom_renderer.rb
def render_shortcode(name)
original_setting = ActionView::Base.annotate_rendered_view_with_filenames
ActionView::Base.annotate_rendered_view_with_filenames = false
result = ApplicationController.render(
partial: "shortcodes/#{name}",
layout: false,
)
ActionView::Base.annotate_rendered_view_with_filenames = original_setting
result
rescue => e
Rails.logger.error("Error rendering shortcode #{name}: #{e.message}")
"{{#{name} /}}"
end
```
Now, if we try again, we should get the banner like we wanted it to:
Everything works, that's good news! But…
Currently, the feature is only useful to render static partials, which may be insufficient for most use cases. Let's learn how to add parameters:
### Self-closing shortcodes with parameters
These are vital when rendering things that might have external IDs: embeds, audio or video players, etc.
We will pass the parameters within the curly braces using a key-value pair for each parameter: `{{you_tube id="123456" /}}`.
To achieve this, we need to modify the regex in the `preprocess` method to capture the params and then parse them using a method to return a hash:
```ruby
def preprocess(document)
document.gsub!(/\{\{(\w+)(.*?)\/\}\}/) do |match|
name = $1
params = parse_params($2)
render_shortcode(name, params)
end
end
```
This new regex adds the `(.*?)` which captures any character zero or more times into our second capture group that's assigned to the `$2` variable.
Using that regex, we would get the following results evaluating `{{alert type="warning"}}`
```ruby
# $1 => 'alert'
# $2 => "type='warning'"
```
Next, we add the `parse_params` function:
```ruby
def parse_params(param_string)
params = {}
param_string.scan(/(\w+)="([^"]*)"/).each do |key, value|
params[key.to_sym] = value
end
params
end
```
We're using the Ruby `scan` method, which can receive a string or a regex to find matches for the patterns it receives in a given string and produce an array of matches.
The regex `/(\w+)="([^"]*)"/` has two capture groups: the first is `/(\w+)` which matches one or more word characters. The second one is `([^"]*)` which matches any character that's not a quote occurring zero or more times.
The second capture group is preceded by a literal `="` and followed by a literal quote `"`.
The method now produces the desired params hash, which we are now passing to the `render_shortcode`:
```ruby
def render_shortcode(name, params)
original_setting = ActionView::Base.annotate_rendered_view_with_filenames
ActionView::Base.annotate_rendered_view_with_filenames = false
result = ApplicationController.render(
partial: "shortcodes/#{name}",
locals: {params: params },
layout: false,
)
ActionView::Base.annotate_rendered_view_with_filenames = original_setting
result
end
```
After this, we can create a partial that requires a parameter and use that in the content to see that everything is working fine.
So a YouTube embed sounds like a good example for this:
```erb
<%# app/views/shortcodes/_you_tube.html.erb %>
```
Now, after we use the shortcode in the content, we should get:
We've made quite some progress. But we're missing the ability to add long form content to the shortcode to display things like callouts, testimonials, rich quotes, etc.
Let's build that now:
### Block shortcodes
The process of adding block shortcodes is very similar, except we will add another regex to capture the content inside the curly braces.
The syntax for this type of shortcodes is `{{shortcode key="value"}}The content goes here{{/shortcode}}`.
So, we now have to add another regex to the `preprocess` method. This regex has 3 capture groups: **the shortcode name, the params and the content.**
```ruby
def preprocess(document)
## Matches self closing shortcodes with params:
## {{ youtube id="1234567890" /}
document.gsub!(/\{\{(\w+)(.*?)\/\}\}/) do |match|
name = $1
params = parse_params($2)
render_shortcode(name, params, nil)
end
## Matches shortcodes with content:
## {{callout type="success"}}Alert content{{/callout}}
document.gsub!(/\{\{(\w+)(.*?)\}\}(.+?)\{\{\/\1\}\}/m) do |match|
name = $1
params = parse_params($2)
content = $3
render_shortcode(name, params, content)
end
end
```
The first two capture groups are the same we used for the past example. The third one is using `(.+?)` which matches one or more characters that are within the opening and closing tags which is essentially the content.
The other change is in `\{\{\/\1\}\}` which states that the closing braces should include the `name` of the shortcode to match correctly.
Lastly, the `/m` at the end modifies the behavior of the `.` in the regex and makes it capture a multi-line content, which is what we will usually pass to the shortcodes.
Now, we need to add the `content` argument to the `render_shortcode` method, and we parse it by calling the `@markdown.render` method on it and then passing the result to the partial.
```ruby
def render_shortcode(name, params, content)
original_setting = ActionView::Base.annotate_rendered_view_with_filenames
ActionView::Base.annotate_rendered_view_with_filenames = false
parsed_content = content&.present? ? @markdown.render(content) : nil
result = ApplicationController.render(
partial: "shortcodes/#{name}",
locals: {params: params, content: parsed_content },
layout: false,
)
ActionView::Base.annotate_rendered_view_with_filenames = original_setting
result
rescue => e
Rails.logger.error("Error rendering shortcode")
end
```
After this, we will add a `notice` partial:
```erb
<%# app/views/shortcodes/_notice.html.erb %>
```
And now, when we add it to the post content, and we should get the following result:
We could also pass a parameter `{{notice title="Remember this"}}Hello from the info shortcode{{/notice}}`, include the parameter in the partial:
```erb
<%# app/views/shortcodes/_notice.html.erb %>
```
And we get the following result:
Currently, the feature is working as we expect, and we should be able to use it to add as many partials as we want, and it should cover almost any scenario.
Of course, your way of implementing the feature can vastly differ from what we showed here. But, all in all, the principles are the same, and you should be able to translate them to fit your needs.
Let's demonstrate that by adding a [VitePress styled shortcodes or callouts](https://vitepress.dev/guide/markdown#custom-containers) and make them work
## VitePress styled shortcodes
VitePress includes a series of so called custom-containers, which are basically a type of Markdown shortcode just like the ones we added before.
They come in 5 flavors: **info, tip, warning, danger and details.**
The first 4 are just regular alerts or callouts and the last one uses the `details` HTML element to show a disclosure widget that expands when clicked and contracts when clicked again, just like an “accordion” component.
The way VitePress defines this custom-containers is the following:
```markdown
::: info
This is the content that's passed to the interior of the callout
:::
::: details
Are you aware that 80% of 100 is 80?
:::
```
So, following what we already did, adding them is a matter of adding a Regex that can capture what we require out of these shortcodes.
Let's add it to the `preprocess` method:
```ruby
document.gsub!(/^:::[ ]+(\w+)\s+(.*?)\s+^:::/m) do |match|
name = $1
content = $2.strip
render_shortcode(name, {}, content)
end
```
This regular expression is evaluated after the first two, which means that it only gets read if the shortcode doesn't match the curly brace style.
It has two capture groups to pick the shortcode name and the content.
Now, if we pass the `:::info` or the `:::warning` code block, we will get the corresponding partials rendered:
Lastly, we add the `details` partial to add that feature too:
```erb
<%# app/views/shortcodes/_details.html.erb %>
Details
<%= content.html_safe %>
```
Now, if we add the shortcode to our content, the result should look like this:
If you want specific callouts to render different partials, you can always namespace the partials to handle them differently. You will just have to create different partials under different namespaces like `vite_press/info` and so on.
Now, to finish the article, we will add GitHub flavored callouts by adding another regex:
## GitHub-flavored alerts
The last part of our shortcode feature involves adding the GitHub-flavored alerts to render as shortcodes or callouts.
The basic format of a GitHub alert is:
```
> [!CAUTION]
> Take into consideration that this is the content for my NOTE alert.
```
They also come in 5 flavors: **NOTE, TIP, IMPORTANT, WARNING and CAUTION**.
This means that we can have a name spaced partial for every one of them, so we can pass the flavor to the `render_shortcode` method.
To simplify things, we will just render the partial that corresponds to the callout name. But you can namespace the rendering to better handle this feature.
The first step is to add the right regex to the `preprocess` method:
```ruby
## Matches the GitHub flavored alert shortcodes (must come first):
## > [!NOTE]
## > Content
document.gsub!(/>\s*\[!(\w+)\]\s*\n((?:>.*\n?)*)/m) do |match|
name = $1.downcase
content = $2.gsub(/^>\s*/, '').strip
render_shortcode(name, {}, content)
end
```
This regex has two capture groups: the shortcode name and the content.
It basically captures a line that starts with `>` has a whitespace followed by one or more words which constitute the shortcode name, followed by a space and a newline that contains any multi-line content and ends with a newline character.
If we want to imitate the GitHub alerts, we would need to add a partial for each one of them and namespace them if they collide with the shortcodes we already have.
After adding a GitHub callout to the content, we get something like this:
## TL;DR
The Marksmith editor is a simple, yet powerful, Markdown editor built for Ruby on Rails applications.
It comes with many features that allow us to concentrate on the content instead of thinking about the formatting.
We can add a shortcode/callout feature to this editor using a custom `redcarpet` renderer, regular expressions and run-off-the-mill Rails partials.
There are many things we have to consider the content before adding the shortcode feature but, all in all, it's about refining the Regex to make it do what we want.
In this article, we added custom, VitePress and GitHub flavored shortcodes to demonstrate how we can add this feature independent of the format we choose to produce the shortcode templates.
I hope you got to better understand the Marksmith editor and how we can integrate a shortcode feature to it.
Have a good one and happy coding.