Categories
Software Development

WordPress REST API Snafus

While migrating data I was forced into evaluating whether using the CLI or REST API would be more effective. If I was smart I would have documented the difficulties I discovered while investigating, but since I didn’t, this is a rough approximation of all the issues I found (or caused).

It all started with a migration I’m running on my wife’s blog. The site I’m migrating used an old PHP gallery system and I’m attempting to import all the imagery into the stock WordPress media library. Because the migration from custom plugins has been painful, the changes need to be as future-proof as possible to prevent the same thing from happening again. The only current custom code I’m using is implementing a custom taxonomy tagging the media with an “album” name.

So my migration process is to: (✅ are done)

  • ✅ scan a post
  • ✅ generate a list of the images that need to be migrated
  • 🔴 upload the images
  • 🔴 Attach to the post (and hopefully tag them with the correct album).
  • A final step of verifying the new content needs to be done manually just because I want to make sure everything looks good for each post.

First attempt: CLI

Locally you can use the WP-CLI tool to upload media to your site quite easily. If you connect to your installation remotely (i.e. wp --ssh=<blah>), this doesn’t work unless you first upload the media to a staging area, use the remote system paths and then remotely clean your staging area.

# Example (--porcelain returns the post_id of the new media item)
wp media import <path-to-file> --post_id=$POST_ID --porcelain

Staging the images would have been the quickest method, and I played with the idea of a shell script, but as I thought about it, I thought using the API would be cleaner (and it should be rock solid since it’s been in use for a couple years all the pain points have probably been smoothed out and answered online, right?).

Using the REST API

Prototyping should be quick, just fire up Postman or use curl to get started. Of course before you can do anything really interesting you need to authenticate yourself. But passing your password every request seems like a recipe for disaster. There has to be a better way!

OAuth

There are a couple plugins that add OAuth (with different flavors and free/pro offerings). Since I wanted as close to the “stock” experience as possible I went with the 1.0a implementation of the core WordPress REST team.

I generated my tokens and keys using Python as the existing OAuth clients for WordPress wouldn’t work for me and don’t seem to be updated to work with newer versions of PHP/composer. (Is this entire system even supported anymore? The WP-CLI client github project was archived in 2019!)

I then spent (not joking) at least 2 hours trying to get the undocumented media endpoint working. Seriously, look at the official documentation and try to tell me how you send the file. These auto-generated docs are missing a lot of information that I had to discover in forum posts and through reading the code.

Eventually I came to the conclusion that I was calling the endpoint correctly (Reader, use a multipart/file POST with the field named ‘file’ — it doesn’t accept multiple files — other parameters as documented above do work. Unfortunately I was also looking at the arguments returned from the REST options call which are NOT fully supported.)

The problem actually lied in the authentication mechanism.

App Specific passwords

Just use this, or you can dive into the OAuth server plugin and fix that… (Please only use if you have HTTPS enabled).

What’s left

Authentication sorted, I dove into the issue of my custom taxonomy which while actually listed in the REST API endpoint as a valid argument, but actually doesn’t do anything. Again, punting here, I assume it’s a loading issue with the taxonomy code, but I didn’t want to dive too deep and I already spent too much time on this. I’ll just send another REST call adding the taxonomy to the post ID.

REST API example

Conclusion

If I could send a message back in time I would probably tell myself to just deal with the CLI method. But since I’ve done so much investigation I’ll probably implement a version of the REST API implementation as it’s more flexible than trying to parse some of the CLI commands (which can have elements oddly named and under odd sub-commands with options that might or might not be documented).

Categories
Software Development

`wp-now` → Awesome

Check out the cool new tool for doing WordPress development. This uses a WASM version of PHP to serve/host the environment directly in your browser. (It does need node for getting started though.)

Categories
Software Development

WordPress Blocks: Hydrate InnerBlocks From HTML

Quick tip about creating or replacing InnerBlocks from HTML in the WordPress block editor. For instance if your block pulls information from a remote API and you want to display as part of the post, or embed in the post such that it’s editable. The block editor (gutenberg) already handles the job very simply.

Use pasteHandler. It not only includes HTML support, but plain text & markdown also!

import { pasteHandler } from '@wordpress/blocks';

And from a dispatch of the block-editor store, use the replaceInnerBlocks() method.

I had a couple issues with the code that made it somewhat confusing.

  1. I’m still getting used to the concept of React hooks, so managing the useSelect, useDispatch, and useCallback might not be 100% correct — but this incantation worked for me 😆.
  2. Originally I tried updating the first InnerBlock and finding it was slightly tricky. Good news — this is actually fairly simple with useSelect to getBlocksByClientId() and then just get the innerBlocks property of the first block returned.

InnerBlocks Example:

// `clientId` is passed as a prop to the edit component.
const { replaceInnerBlocks } = useDispatch( 'core/block-editor' );

// call `updateInnerBlocks( markdownOrHTMLContent );`
const updateInnerBlocks = useCallback(
	source => {
		// Get an array of inner blocks
		const newInnerBlocks = pasteHandler( {
			HTML: '',
			mode: 'BLOCKS',
			plainText: source,
		} );
		replaceInnerBlocks( clientId, newInnerBlocks );
	},
	[ innerBlocks ]
);

Interested in more? See other posts about WordPress and the block editor.

Categories
Software Development

MermaidJS & WordPress

I get confused by a lot of words. The combination of complex ideas and large walls of text short-circuit and crash my brain. I’ll have to read and re-read a text-only passage multiple times to understand exactly what’s being described. Speaking presentations exacerbate the issue.

However the adage “A picture is worth a thousand words” definitely applies to me. Given graphical clues demonstrating the principle helps me begin to make connections and jumpstarting my understanding.

graph TD
   start(Explain an idea) -->|with words|start
   start --> |with pictures| understanding

One of the tools I’ve found very helpful (if only for myself) is MermaidJS. This software lets you create several different types of charts using a markdown-ish syntax. For instance, the above chart was created with:

graph TD
   start(Explain an idea) --> |with words| start
   start --> |with pictures| understanding

This grants other benefits in addition to the quicker comprehension. Quick edits to the text automatically recreates the chart without requiring any manual intervention/tweaking to the rest of the diagram.

MermaidJS supports flowcharts, sequence diagrams, class diagrams, state diagrams, ERD (entity-relationship diagrams), Gantt charts, pie charts, requirement diagrams and User Journey diagrams.

On my last post I actually used it to show the sequence of flows between WordPress, WooCommerce and webhooks.

WordPress Plugins

Working at Automattic, all our internal documentation and institutional knowledge is recorded using WordPress (you should really checkout using p2’s — it’s an awesome tool). Using Mermaid in this environment has been fantastic — Mermaid can be supported anywhere you have HTML (and I can be exported to SVG too), but as WordPress is running 40% of the web, this seems like a pretty good place to start.

In fact MermaidJS has a page dedicated to integrations of their software and mention a couple plugins:

  • WordPress Markdown Editor (seems to have been rebranded to WP Githuber-MD — see below)
  • WP-ReliableMD: The page was in a different language than I understand and I made a snap decision to avoid it 😢. Should I revisit this decision?

Here is my un-asked for 2¢ about some different WordPress plugins supporting MermaidJS.

WP Mermaid (and WP Githuber-MD)

These plugins are both created & maintained by Terry Lin. I think they’re both great. They support Gutenberg blocks. They display the Mermaid text. Give them a shot and see if they float your boat.

Personally, I didn’t need the full markdown editor, and WP Mermaid had a couple small annoyances that I couldn’t solve. I submitted a couple PR’s, but decided I could use something a little simpler, which brings me to my own plugin…

MerPress

I can’t take full credit for this plugin. My coworker Michael Pretty actually did most of the work on this iteration and I submitted it to WordPress.org. I’m giving him full-credit. He did an awesome job with it.

A couple features I really like:

  • MerPress keeps the wptexturize functionality. Posts preserve the capability to convert simple characters like quotes and dashes into the nice curly quotes and dashes (e.g. “”, –, —).
  • In the Gutenberg editor, the block is encased in a code block letting you edit the code with a monotype font.

Some features I would like to see:

  • Ability to change the theme site-wide and per block
  • Add other configuration parameters to the block
  • Support 3rd party CDN (something that WP Mermaid already does)

Conclusion

If you learned nothing else, you now know I need pictures in order to comprehend complex concepts. Hopefully it’s also obvious that Mermaid is a neat tool and can be used to quickly share intention and purpose and thoughts without all the words.

graph TD
  ad(all done) --> te(the end)

Please let me know if I’m missing a plugin that I should try or if there is something you would like to see added to MerPress…

Categories
Software Development

Working with WooCommerce Webhooks

I recently went deep into the WooCommerce Webhooks code. I’m preserving my thoughts here (if anyone needs more help — feel free to comment — I’m happy to share what I know).

Webhooks?

TL;DR; A separate website can be notified when you make a sale by creating a webhook.

WooCommerce provides a method to send announcements of actions that occur on your site to any URL. These announcements (topics) consist of WooCommerce domain objects (orders, products, subscriptions, etc. also generically named resources) and include their life-cycle events (creation, updates, deletion, etc.). The WooCommerce documentation has a pretty good writeup about this.

WooCommerce Webhooks Handling Overview

sequenceDiagram

participant WordPress
activate WordPress
WordPress ->> WooCommerce: load woocommerce
activate WooCommerce
WooCommerce ->> Webhooks: instantiate webhooks
activate Webhooks
loop Every webhook
Webhooks ->> Webhooks: hook topics
Note right of Webhooks: An topic is defined by a <br> resource and an event.<br>-<br>A topic has a mapping to one or <br>more WordPress hooks. When the <br>hook fires (i.e. do_action),<br> the webhook is enqueued.<br>-<br>See $webhook->enqueue()
end
deactivate Webhooks
deactivate WooCommerce
WordPress ->> WordPress: ...
WordPress ->> Webhooks: Webhook Event (e.g. Order created)
Note right of Webhooks: Events are added to a global<br>array with: 1️⃣ the Webhook object<br>and 2️⃣ the args passed to the hook.<br>-<br>See $webhook->process()
WordPress ->> WordPress: ...
Note right of WordPress: shutdown
loop Enqueued Webhooks
WordPress ->> Webhooks: deliver
Note right of Webhooks: Actually 2 different methods to <br>deliver. Defaults to async (using<br> Action Scheduler), but can<br>run synchronously.<br>-<br>See $webhook->deliver()
end
deactivate WordPress

Using Webhooks For Fun and Profit

Topics aren’t limited to just WooCommerce domain objects. You can use any WordPress hook to send a webhook. Using the resource “action”, add the hook name as the event and you will receive the first argument passed to the hook as the parameter in your webhook. Events must start with wc_ or woocommerce_.

Example Payload

{
  "action":"woocommerce_my_custom_event",
  "arg":{
     "test":"something I passed in"
  }
}

Deliver custom topics

Using the action resource is limiting.

  • The hook name needs to have a special prefix
  • The shape of the data returned will be wrapped in the arg parameter.
  • WooCommerce uses ActionScheduler for storing/managing async events, capping the size of the data at 191 characters. (Async handling can be disabled to avoid this limitation).

What I really want is my own resources/events. For this example let’s pretend we needed to add our own bookmark object, and we want to send a webhook whenever a bookmark is created.

First, add our topic to the list so it can be appropriately enqueued. This mapping is from topic → hook. When the hook is called (e.g. do_action( 'send_bookmark_created_webhook', $bookmark->id ); ) The webhook is added to the queue.

/**
 * Add webhook topics.
 *
 * This is used in the startup portion of webhooks, and is called for each webhook.  As the webhook has the reference
 * to the specific topic, just return the WordPress action to listen for this topic.
 *
 * @param array $topic_hooks
 * @param \WC_Webhook $webhook
 */
function add_topics( $topic_hooks, $webhook ) {
	$topic_hooks['bookmark.created'] = 'send_bookmark_created_webhook'; // When we create a bookmark, we'll fire this action.
	return $topic_hooks;
}

add_filter( 'woocommerce_webhook_topic_hooks', 'add_topics', 10, 2 );

Now we need to create our payload (avoiding the 191 character limit).

/**
 * Build a payload for our custom topics.
 *
 * @param $filtered_payload mixed  Passed from the existing build_payload function.
 * @param $webhook_resource string Resource parsed from the webhook topic.
 * @param $arg mixed               Args passed to the trigger for the topic.
 * @param $webhook_id int          Can use this to rehydrate a copy of the webhook (e.g. new WC_Webhook( $id )).
 *
 * @return mixed
 */
function build_payload( $filtered_payload, $webhook_resource, $arg, $webhook_id ) {
	if ( 'bookmark' === $webhook_resource ) {
		return get_bookmark( $arg );
	}
	return $filtered_payload;
}

add_filter( 'woocommerce_webhook_payload', 'build_payload', 20, 4 );

Finally, whitelist the webhook.

/**
 * Before the webhook is sent this hook determines if it should be delivered.
 *
 * Have to override this because the custom types.  Alternately could continually overwrite
 * `woocommerce_valid_webhook_resources' and 'woocommerce_valid_webhook_events` like is done
 * in the store currently.
 *
 * @param $should_deliver bool Existing value
 * @param $webhook \WC_Webhook
 * @param $arg mixed           Delivery body
 */
function should_deliver( $should_deliver, $webhook, $arg ) {
	if ( 'bookmark.created' === $webhook->get_topic() ) {
		return true;
	}
	return $should_deliver;
}

add_filter( 'woocommerce_webhook_should_deliver', 'should_deliver', 10, 3 );

At the end of the day

If this helped you or if something remains confusing let me know, I’m interested in hearing people’s pain points and wish list items. Personally, I’m looking at adding retry functionality to this infrastructure and will add another post when I get that sorted.