XF 2.4 Tiptap: A new editor for XenForo

2.4 editor flare.webp
Rich text editors are hard. When you drill down into how they actually work, it's a wonder why there are so many of them, and why developers would willingly spend a significant part of their life working on one. At XenForo we have been through ... counts on fingers ... three different rich text editors in our history, so if we're being generous we can at least attest to what a feat of engineering they are and how difficult they can be to get right -- and all we have to do is implement them.

That's no easy feat either, though. We have an increasing number of features that we need from an editor, we have a layer of conversion from HTML to BB code (and back again), and we have an overall user base of millions of members across all of our customers who all seem intent on breaking our editor in weird and wonderful ways. So picking a new editor is fraught with risk and challenges. Sometimes, even when we pick the right editor, it doesn't always stay that way.

Over recent years we've been keeping a close eye on the developments within the rich text editor space and while there are a few interesting projects, there is only one very clear winner, and that is Tiptap.

What is Tiptap?​

Tiptap is a completely modular, open source and headless editor framework. Headless means that there is no provided user interface. Modular means you can enable only the functionality that you need. It is so modular that, technically, you don't even have to enable paragraphs or text (though that might impede its usefulness). It also has a fantastic extension system that allows us to not only create our own extensions, but also extend existing extensions! (more on that later).

Tiptap, which is itself powered by the amazing ProseMirror toolkit, does not simply attempt to decorate input with arbitrary HTML like most rich text editors. Rather, it has a very strict (but extendable) schema made up of Marks and Nodes. Marks are inline formatting (such as bold and italic) and Nodes are block-level formatting (such as paragraphs, quotes, and code blocks).

Your first look​

Strap in, because there's lots to show you. But first, an artist's representation of how it might look in either light or dark styles:

2.4 editor gradient.webp


The overall look and feel of the editor isn't 100% finalised at this point, but the current iteration, as pictured above, does lean into being familiar. One of the notable changes currently is that we have moved the toolbar to the bottom of the editor, making the controls more accessible, particularly on smartphones.

Want it at the top? No problem, that's a single line of CSS to make that happen.

The toolbar is, as ever, managed entirely through the existing editor button manager in the admin control panel, so the toolbar itself can be made as full-featured or as minimalist as you like.

Great for users and power users alike​

For a lot of things, you might not even need to use the toolbar. The brave amongst you might even choose to disable the toolbar entirely, thanks to the simplicity with which many editor features can be used.

As part of the overall schema that powers Tiptap, "input rules" can be defined by developers for each extension, which, in short, are "Markdown-style" editing:



Power users will not only be able to use Markdown-style syntax to produce content - which is enabled for a significant amount of editor functionality - but they will also be able to see the formatting change in real-time.

What you see is really what you get​

In fact, all core editor functions have a visual representation of their content, whereas they might not have previously. We are putting an end to seeing raw BB code syntax in the editor (unless you type it yourself, which still works):



This level of visual clarity for what your final content will look like extends even to code blocks, which are now fully syntax highlighted in the editor. The syntax highlighting is provided by highlight.js. As a result, we have upgraded our BB code syntax highlighting for code blocks to also use highlight.js going forward. We are including more languages (and more relevant languages) by default, and it remains easy to add additional ones to fit your particular niche.



It truly is a "what you see is what you get" experience, and to underline that even more, wait until you see what happens when you paste an embeddable link (best viewed full screen):



A consistent UI powered by XenForo​

As we mentioned earlier, Tiptap is a completely headless editor, meaning we have built the entirety of the UI that you can see. This is now more consistent and more in line with the default XenForo styling than ever. With our previous editors, there was always a hybrid of different UI pieces: some from the editor itself, others that we bolted on. For example, in our current editor implementation there are three distinct UI components we use - Froala pop-ups (insert link), overlays (insert media) and menus (emoji). With the new editor, everything you see is crafted from default XenForo functionality and styling, ensuring consistency:
CleanShot 2024-12-17 at 16.36.31.gif


By the way, the "Insert image" menu of the editor now supports multiple file uploads, including when dragging and dropping!

An exciting experimental editing experience​

While the new UI for the new editor will be very familiar to most, because we now have full control over what the editor can do, we are also experimenting with more innovative methods for writing content. Let's have a sneak peek now at what that might look like (best viewed full screen):



We haven't yet fully decided what form this new editing experience will take. It might be something that is user-selectable, admin-selectable or something that can be enabled as a hybrid with the toolbar. Your thoughts on this approach are welcome!

Extending the editor​

All of this additional functionality is possible, not only with the strong foundations provided by both ProseMirror and Tiptap, but also through an extremely flexible and well thought out system enabling us to build our own functionality. By virtue of that, add-on developers are going to have a great experience bringing new features to our new editor.

For those of you less focused on development, this might not be quite as interesting. But let's just take a look at an example extension that adds some functionality to the editor:



While this is not (currently) shipping as a built-in feature (we haven't even created an associated BB code to render it in posts), we wanted to share it as an example of how easy and intuitive Tiptap is. Here's the sample Tiptap extension code we wrote that powers the functionality seen in the previous video:

JavaScript:
XF.EditorExtensions.registerExtension(({ Node, InputRule }) =>
{
    const inputRegex = /^!!!(highlight|important|success|warning|error)[\s\n]$/

    return Node.create({
        name: 'blockMessage',

        group: 'block',

        content: 'block+',

        defining: true,

        addAttributes()
        {
            return {
                style: {
                    default: 'highlight',
                    parseHTML: element => element.getAttribute('data-style'),
                    renderHTML: attributes =>
                    {
                        return {
                            'data-style': attributes.style,
                        }
                    },
                }
            }
        },

        parseHTML()
        {
            return [
                {
                    tag: 'div[data-type="block-message"]',
                },
            ]
        },

        renderHTML({ node })
        {
            return [
                'div',
                {
                    'data-style': node.attrs.style ?? 'highlight',
                    'class': 'blockMessage blockMessage--iconic blockMessage--' + (node.attrs.style ?? 'highlight'),
                },
                [
                    'div',
                    {},
                    0,
                ],
            ]
        },

        addInputRules()
        {
            return [
                new InputRule({
                    find: inputRegex,
                    handler: ({ state, range, match }) =>
                    {
                        const { from, to } = range
                        const tr = state.tr

                        const node = this.type.createAndFill({
                            style: match[1]?.trim(),
                        })
                        tr.replaceRangeWith(from, to, node)
                    },
                }),
            ]
        },
    })
})

A full breakdown of exactly how this works is beyond the scope of this post, but feel free to spend some time understanding it and the editor docs. If you have existing editor-based add-ons, the great news is that as long as you have some basic understanding of JavaScript, you should be able to produce your own extensions. We are not implementing any additional frameworks; everything is based on vanilla JavaScript.

One more thing...​

The astute amongst you might have already noticed that new lines work differently in the new editor compared to the current one, and there's a very good reason for that. We are pleased to announce that, finally, we are going to be using paragraph (<p>) tags properly starting with XenForo 2.4.

Hitting enter/return in the new editor now creates a new paragraph (with appropriate spacing). This may take a few moments to get used to, as hitting enter/return will now produce new line spacing equivalent to pressing enter/return twice currently. You can insert a "hard break" (producing a <br /> tag) with Shift + Enter. While this might require a brief adjustment, it is now consistent with how many applications and office suite tools work, so we don't expect it to take long.

The correct use of paragraphs doesn't stop with the editor, as we now correctly render content with paragraphs too!

HTML:
<h2 class="bbHeading">A brand new editor experience!</h2>
<div class="bbImageWrapper " title="Happy Season 9 GIF by The Office">
    <img src="https://media3.giphy.com/media/o75ajIFH0QnQC3nCeD/200.gif" class="bbImage">
</div>
<p>You might <b>recognise</b> the concept from applications such as <a href="https://notion.so" target="_blank">Notion</a>. You can add nodes from anywhere!</p>

<p>You can duplicate nodes. And move them around!</p>

<blockquote>
    <p>I've never had so much fun in an editor!</p>
</blockquote>

We haven't even changed the underlying BB code syntax. This is achieved with a brand new BB code parser which intelligently detects the correct boundaries for paragraphs and is able to distinguish between line breaks and paragraphs.

More coming soon!​

Thank you for joining us on this first look into the new editor we're implementing in XenForo 2.4. We're excited to get this in your hands in 2025. In the meantime we have some additional things for the editor that we're still wiring up:
  • Improvements to the developer experience for implementing custom editor functionality, BB code and custom buttons
  • Making use of the new editor elsewhere in XenForo
  • Further improvements to the UI and UX
  • Finalising the introduction of the experimental experience as an option
This will be the final Have you seen...? thread for 2024 and we will be back in the new year with even more to show you as we approach the final stages of this release.
 
I don’t suppose it could be user choice?
You can add a custom user field checkbox and group it under preferences, and then enable or disable the line of css to move the bar to the top depending on whether or not they've enabled that preference!



This all looks great! I do hope there will still be an option to toggle display of the raw input, however, as pasting things into WYSIWYG editors can cause unexpected behaviours and there'll likely be cases where users want to paste in text into a plaintext editor first and then enable WYSIWYG, or they might want to delete some formatting but keep the text and other parts of the formatting and that might be more easily done with the raw text and so on and so forth. There's definitely various use cases where having access to the raw text is important for power users.

Answered right before I posted! Great news, thank you Chris D :)
 
As long as we don't have to have Markdown enabled, I'm all for it. As someone who's written decades (likely before some here were even born) in familiar word processing applications, my preferred writing experience (as well as most of the colleagues I work with) is to not have my writing altered by extra characters typed along with the text. I seem to recall that Markdown was enabled for the current editor, but at some point that was changed. Hoping we can turn it off if that's the case.

As long as our keyboard shortcuts remain (Ctrl-B for bold, Ctrl-I for italic, etc. as word processors have done for decades), I'm good with a new editor.
This is a major concern here. Markdown just gets in the way in 99% of our use cases. Please include an option to disable for individual posts when it just prevents you from typing normally. The whole point of bbcode is that it doesn't get in the way.
 
This is a major concern here. Markdown just gets in the way in 99% of our use cases. Please include an option to disable for individual posts when it just prevents you from typing normally. The whole point of bbcode is that it doesn't get in the way.
Switch to BB code if it gets in the way. I don’t see how it would though. Whatever markdown style input rules get applied, get applied live. So you would just adjust it if needed before posting.

Not like now where the formatting gets applied invisibly on save.

It’s a wysiwyg editor. If you don’t like what you see, whatever you need to do to fix it will be obvious and simple.
 
Switch to BB code if it gets in the way. I don’t see how it would though. Whatever markdown style input rules get applied, get applied live. So you would just adjust it if needed before posting.

Not like now where the formatting gets applied invisibly on save.

It’s a wysiwyg editor. If you don’t like what you see, whatever you need to do to fix it will be obvious and simple.
Perhaps I misunderstood your previous reply, but it sounds like you said it wasn't optional earlier in the thread and would be unlikely to be made so in this reply. Now you are saying it can be switched to a bbcode mode only? I'm sorry I'm just not following, which is it?

The problem with markdown in a WYSIWYG editor is precisely the need to "fix it" every time I so much as have two underscores or asterisks anywhere, or whatever other typical forum/word processor formatting we're all used to suddenly gets a new meaning. I deal with it all the time on Reddit, even with a live preview from RES, it sometimes takes more time for me to "fix" what I typed to just work than it did to type the post out initially, because I have to find every little instance of "markdown" that wasn't intended to be markdown and find out some way to bypass it. And these types of posts are like text messages compared to what you can see in a forum thread in terms of complexity. Some people go to extreme lengths to format their posts using ASCII art and the like. That will all be broken the moment this gets implemented, or the moment the editor is open on such a post if it is not automatic with the new parser (I certainly hope it's not).

People like you and I can probably figure it out and work around it. It'll take time, but we'll learn to do so one way or another. The problem comes when other, less tech-savvy users are doing the same thing they always did to suddenly find it produces a different result. And what if someone edits a post that has text which can be interpreted as "markdown" but wasn't a problem with the old editor? I can easily see people going in and making a quick edit to a large post with complex formatting, and not even realizing something else changed when they hit "save" because they simply weren't looking where the change might have occurred. Then it causes a support nightmare for us staff because we start getting reports that posts are "broken" and we have to figure out and explain what happened.

If it indeed can be disabled on a per post (ideal), per person (less ideal), or global basis (not ideal, but manageable), then that's fine as long as we can turn it off. If not, then I'm sorry, but this is a "feature" that's going to create more problems than it solves and create extra work for us, pulling precious resources away from other projects to be support triage instead. In that case we may be forced not to upgrade to the latest at all, despite all the other great optimizations and features that are being introduced.

I'm sure the Xenforo dev team worked very hard to make this happen, and I don't want to devalue the effort here. That said it's very important that the introduction of a new editor is as transparent and seamless as possible, otherwise it can have wildly complicated downstream impacts. This is where user choice has to come into play - give people the option to disable it when it becomes a headache.
 
Markdown is not optional. Switching to the raw bb code editor (as you can do now) is still possible.

Provide some examples and I’ll test them. I really don’t think it will be an issue.
 
This is great news. Froala is knackered.

For a thicky like me, would the extensions allow us to choose a markup input regex that inserted coloured font, just as you would font for italics?
 
Last edited:
A developer could implement or change the input rules regexes as they see fit, yes. So maybe you could do !some text! to get red text or similar.
 
Would be great if this editor is used and available everywhere in the Admin CP that you use an editor. Sometimes the editor supports BBcode, etc, and sometimes not. That's kind of annoying.
 
Well, the editor is only used by default for content creation. We don’t really use it anywhere in the admin control panel.

We do have a mix of places that support HTML though and that has to be typed manually.
 
i might have missed this in first post... but are you unfurling urls as they are entered in editor as well? just curious. what happens if i just copy paste bbcode or markdown from an external editor. i guess it is too early for these questions. can only see how this affects my workflow when a beta is released. this is bigger than how it appears to be. this is basically putting gutenberg in wordpress comments section. regular folks on the web might be used to word processors but not everyone is going to be used to a notion like content creation experience (Word does not embed tweets or youtube videos or render links when you just paste a link for instance). i mean this makes sense for article threads maybe. but it appears too much for general threads that forums are generally known for. in the end, i guess, this is what we are getting. so i am going to well... just wait for rollout here and hope for the best. 🤞
 
Markdown is not optional. Switching to the raw bb code editor (as you can do now) is still possible.

Provide some examples and I’ll test them. I really don’t think it will be an issue.
I can pretty easily provide an example of auto-parsing markdown being frustrating to work with:

If I am to write the emoticon ^_^ twice in one line, e.g. Hi guys ^_^ I hope you're doing well! My week's been busy and a little overwhelming, but I've learnt a lot! ^_^ or something idk, markdown parsers will mistake this for me doing _this_ and turn that message into "Hi guys ^^ I hope you're doing well! My week's been busy and a little overwhelming, but I've learnt a lot! ^^" which of course, is not what I wanted.

From looking at the previews you've provided, the markdown markup vanishes and is replaced with the WYSIWYG formatting right? There would therefore need to be a way for me to display the raw format in order for me to then escape those underscores and prevent unintentional formatting taking place.

EDIT: well! it went ahead and did the exact error I was describing LOL let me try and fix it give me a minute to edit the post

EDIT2: There we go
 
I can pretty easily provide an example of auto-parsing markdown being frustrating to work with:

If I am to write the emoticon ^_^ twice in one line, e.g. Hi guys ^_^ I hope you're doing well! My week's been busy and a little overwhelming, but I've learnt a lot! ^_^ or something idk, markdown parsers will mistake this for me doing _this_ and turn that message into "Hi guys ^^ I hope you're doing well! My week's been busy and a little overwhelming, but I've learnt a lot! ^^" which of course, is not what I wanted.

From looking at the previews you've provided, the markdown markup vanishes and is replaced with the WYSIWYG formatting right? There would therefore need to be a way for me to display the raw format in order for me to then escape those underscores and prevent unintentional formatting taking place.

EDIT: well! it went ahead and did the exact error I was describing LOL let me try and fix it give me a minute to edit the post

EDIT2: There we go
Doesn’t seem to be an issue with Tiptap no.

IMG_2077.webp
 


Write your reply...
Back
Top Bottom