Categories
Software Development

WordPress REST API Snafus

While migrating data I was forced into evaluating whether using the CLI or REST API would be more effective. If I was smart I would have documented the difficulties I discovered while investigating, but since I didn’t, this is a rough approximation of all the issues I found (or caused).

It all started with a migration I’m running on my wife’s blog. The site I’m migrating used an old PHP gallery system and I’m attempting to import all the imagery into the stock WordPress media library. Because the migration from custom plugins has been painful, the changes need to be as future-proof as possible to prevent the same thing from happening again. The only current custom code I’m using is implementing a custom taxonomy tagging the media with an “album” name.

So my migration process is to: (βœ… are done)

  • βœ… scan a post
  • βœ… generate a list of the images that need to be migrated
  • πŸ”΄ upload the images
  • πŸ”΄ Attach to the post (and hopefully tag them with the correct album).
  • A final step of verifying the new content needs to be done manually just because I want to make sure everything looks good for each post.

First attempt: CLI

Locally you can use the WP-CLI tool to upload media to your site quite easily. If you connect to your installation remotely (i.e. wp --ssh=<blah>), this doesn’t work unless you first upload the media to a staging area, use the remote system paths and then remotely clean your staging area.

# Example (--porcelain returns the post_id of the new media item)
wp media import <path-to-file> --post_id=$POST_ID --porcelain

Staging the images would have been the quickest method, and I played with the idea of a shell script, but as I thought about it, I thought using the API would be cleaner (and it should be rock solid since it’s been in use for a couple years all the pain points have probably been smoothed out and answered online, right?).

Using the REST API

Prototyping should be quick, just fire up Postman or use curl to get started. Of course before you can do anything really interesting you need to authenticate yourself. But passing your password every request seems like a recipe for disaster. There has to be a better way!

OAuth

There are a couple plugins that add OAuth (with different flavors and free/pro offerings). Since I wanted as close to the “stock” experience as possible I went with the 1.0a implementation of the core WordPress REST team.

I generated my tokens and keys using Python as the existing OAuth clients for WordPress wouldn’t work for me and don’t seem to be updated to work with newer versions of PHP/composer. (Is this entire system even supported anymore? The WP-CLI client github project was archived in 2019!)

I then spent (not joking) at least 2 hours trying to get the undocumented media endpoint working. Seriously, look at the official documentation and try to tell me how you send the file. These auto-generated docs are missing a lot of information that I had to discover in forum posts and through reading the code.

Eventually I came to the conclusion that I was calling the endpoint correctly (Reader, use a multipart/file POST with the field named ‘file’ — it doesn’t accept multiple files — other parameters as documented above do work. Unfortunately I was also looking at the arguments returned from the REST options call which are NOT fully supported.)

The problem actually lied in the authentication mechanism.

App Specific passwords

Just use this, or you can dive into the OAuth server plugin and fix that… (Please only use if you have HTTPS enabled).

What’s left

Authentication sorted, I dove into the issue of my custom taxonomy which while actually listed in the REST API endpoint as a valid argument, but actually doesn’t do anything. Again, punting here, I assume it’s a loading issue with the taxonomy code, but I didn’t want to dive too deep and I already spent too much time on this. I’ll just send another REST call adding the taxonomy to the post ID.

REST API example

Conclusion

If I could send a message back in time I would probably tell myself to just deal with the CLI method. But since I’ve done so much investigation I’ll probably implement a version of the REST API implementation as it’s more flexible than trying to parse some of the CLI commands (which can have elements oddly named and under odd sub-commands with options that might or might not be documented).

Categories
Software Development

Proposal for Python structs

I really enjoyed this article. Seems like a great idea, but since I didn’t see a good way to register my support on the site I’m putting it here. He has my vote.

Categories
Software Development

`wp-now` β†’ Awesome

Check out the cool new tool for doing WordPress development. This uses a WASM version of PHP to serve/host the environment directly in your browser. (It does need node for getting started though.)

Categories
Software Development

WordPressΒ Blocks: Hydrate InnerBlocks From HTML

Quick tip about creating or replacing InnerBlocks from HTML in the WordPress block editor. For instance if your block pulls information from a remote API and you want to display as part of the post, or embed in the post such that it’s editable. The block editor (gutenberg) already handles the job very simply.

Use pasteHandler. It not only includes HTML support, but plain text & markdown also!

import { pasteHandler } from '@wordpress/blocks';

And from a dispatch of the block-editor store, use the replaceInnerBlocks() method.

I had a couple issues with the code that made it somewhat confusing.

  1. I’m still getting used to the concept of React hooks, so managing the useSelect, useDispatch, and useCallback might not be 100% correct — but this incantation worked for me πŸ˜†.
  2. Originally I tried updating the first InnerBlock and finding it was slightly tricky. Good news — this is actually fairly simple with useSelect to getBlocksByClientId() and then just get the innerBlocks property of the first block returned.

InnerBlocks Example:

// `clientId` is passed as a prop to the edit component.
const { replaceInnerBlocks } = useDispatch( 'core/block-editor' );

// call `updateInnerBlocks( markdownOrHTMLContent );`
const updateInnerBlocks = useCallback(
	source => {
		// Get an array of inner blocks
		const newInnerBlocks = pasteHandler( {
			HTML: '',
			mode: 'BLOCKS',
			plainText: source,
		} );
		replaceInnerBlocks( clientId, newInnerBlocks );
	},
	[ innerBlocks ]
);

Interested in more? See other posts about WordPress and the block editor.

Categories
Software Development

WordPress Blocks: React Context Cuts Clutter

Using Reacts’ Context in WordPress blocks passes parameters painlessly. Attributes carried by context keep coupling minimal.

Reacts’ Context feature facilitates distributing data down a component tree to any descendant component. Instead of passing attributes between your components manually (e.g. <FooComponent attrib={ attrib }>), which tightly couples the current attributes and components, using a block-level context makes it easy to update and manage attributes (and their changes).

WordPress Blocks (a.k.a. Gutenberg) supports a similar concept called Block Context, but it’s for communicating and sharing attributes between discrete blocks (those registered using registerBlockType).

To someone familiar with React the following is probably obvious and straight-forward, but I found implementing it non-trivial, and also not covered in the WordPress documentation. Regardless, this appears to be so functional it should be a best practice.

Example:

A WordPress block has at minimum 2 top-level components, an Edit and a Save. Each component is passed the block attributes in the first argument. Typically the save component is less-interesting as it specifies the block data saved in the post-content, but if required it would function identically to the following example used with the edit component.

For this example let’s examine a block with 1 attribute, passed to both the toolbar and the sidebar.

export default function Edit( props ) {
	const { foo = '' } = props.attributes;
	const blockProps = useBlockProps();

	const updateFoo = useCallback(
		( _foo ) => {
			props.setAttributes( { foo: _foo } );
		},
		[ foo ]
	);

	return (
		<div { ...blockProps }>
			<MyPluginBlockControls foo={foo} onFooChange={updateFoo}></MyPluginBlockControls>
			<MyPluginInspectorControls foo={foo} onFooChange={updateFoo}></MyPluginInspectorControls>
			...
		</div>
	);
}

While this example is a bit contrived (with only one attribute it is probably not necessary to create subcomponents managing our controls), it should be evident that adding additional attributes cascades changes through the tree. Using a Context instead would resemble code like (differences are highlighted):

import { FooContext } from './context';

export default function Edit( props ) {
	const { foo = '' } = props.attributes;
	const blockProps = useBlockProps();

	const updateFoo = useCallback(
		( _foo ) => {
			props.setAttributes( { foo: _foo } );
		},
		[ foo ]
	);

	const blockContext = {
		foo,
		updateFoo,
	};

	return (
		<FooContext.Provider value={ blockContext }>
			<div { ...blockProps }>
				<MyPluginBlockControls></MyPluginBlockControls>
				<MyPluginInspectorControls></MyPluginInspectorControls>
				...
			</div>
		<FooContext.Provider/>
	);
}

The Context

Creating the context itself is pretty simple:

import { createContext, useContext } from '@wordpress/element';

export const FooContext = createContext( {
	foo: undefined,
	updateFoo: () => {},
} );

export const useFooContext = () => useContext( FooContext );

Using the Context in a sub-component

import { useFooContext } from './context';

export function MyPluginBlockControls() {
	// Use `foo` as normal.  If you need to write it, you can also add `updateFoo`.
	const { foo } = useFooContext();
}

Adding additional attributes

If you modify your block.json to have additional attributes it’s simple to add them to the context (use the property attribute and create a write function at the top-level for every attribute ), then it’s trivial to change the components to import the attribute with the const {foo,bar} = useFooContext() syntax.

Conclusion

This pattern appears to be a good way to decouple attributes within a WordPress block that need to be passed either to re-usable components or just to simplify what code should change when adding/updating the attributes being used. I’m still a React novice and I would love to know if there’s a problem or if it could be improved any way — let me know in the comments!

Categories
Software Development

Debugging Gutenberg Blocks: 1

I’m adding some tags for tips as I find them.

I found this tip helpful when debugging what the WordPress Block Editor (Gutenberg) was actually doing behind the scenes.

Find the post id (in the edit url there will be a field for post (e.g. post.php?post=121&action=edit). Using the WP-CLI tool you can quickly view the source of the post (assuming you have a very basic post with just the block you are developing) by running:

wp post get 121 --field=post_content

This will display the content showing the HTML as it is stored in the database — including the attributes for the block!

Categories
Software Development

Filecasted: Create Podcasts From Local Files

Running isn’t the most enjoyable activity, but it’s great for burning calories. One of the best ways to cope is by listening to podcasts. Podcast apps are generally great and provide innovative features — like speeding up the playback (to get through the content quicker). Switching between podcasts and audiobooks becomes jarring after expecting many of the same behaviors, so I spent a weekend throwing together a quick application to create podcasts from local files.

It’s pushed to github (as open-source) and can be used for turning local mp3’s into deliverable podcast episodes. Use it to produce a podcast or publish audio files to the web.

Filecasted

It’s a simple app — it takes a list of files and creates an RSS (.xml) file. The RSS file can be hosted on something like AWS for general delivery. I threw it together using poetry/python, and development is started by:

  1. pip install poetry
  2. poetry install
  3. poetry shell — the filecast command is now available.
$ filecast --help

Usage: filecast [OPTIONS] INPUT

  Create a podcast file from INPUT.

  INPUT can be a playlist (`.m3u`), stdin (`-`), a directory or a collection
  of audio files.  Playlists and stdin should be a simple text document with
  a list of files to add.

Options:
  --version
  -v, --verbose
  -f, --force             Force the output to an existing file.
  --audio-extensions EXT  Comma separated list of processable extensions.
                          [default: .mp3,.mp4,.wav]

  -a, --append            Append INPUT to the output file.
  -o, --output PATH
  -n, --dry-run           Run as normally, but don't make any permanent
                          changes

  --help                  Show this message and exit.

It’s still a bit buggy. For instance, the album art doesn’t show correctly in the podcasts app I use, and some of the audio files downloaded strangely (but I don’t believe that was related to my code).

Also, there is a lot of room for improvement. It would be nice to manage some of the metadata and update/change feeds as needed. Or maybe support transcoding/speeding files up/adding/editing/splicing the audio files… Or adding support for awscli and delivering both the “episodes” and the RSS file to their space on the internet.

The sad ending

Despite the bugs, Filecasted worked for me in the limited capacity that I needed it to. I was able to create podcasts from local files, publish to AWS and then listen to it with my podcast app. Since I last used it though, I switched to Pocket Casts, which has a feature to play local files without the intermediate steps needed to host and download.

So the bugs will likely remain and I’ll just keep the joyful memory of building and using my creation 😒.

Categories
Software Development

MermaidJS & WordPress

I get confused by a lot of words. The combination of complex ideas and large walls of text short-circuit and crash my brain. I’ll have to read and re-read a text-only passage multiple times to understand exactly what’s being described. Speaking presentations exacerbate the issue.

However the adage “A picture is worth a thousand words” definitely applies to me. Given graphical clues demonstrating the principle helps me begin to make connections and jumpstarting my understanding.

graph TD
   start(Explain an idea) -->|with words|start
   start --> |with pictures| understanding

One of the tools I’ve found very helpful (if only for myself) is MermaidJS. This software lets you create several different types of charts using a markdown-ish syntax. For instance, the above chart was created with:

graph TD
   start(Explain an idea) --> |with words| start
   start --> |with pictures| understanding

This grants other benefits in addition to the quicker comprehension. Quick edits to the text automatically recreates the chart without requiring any manual intervention/tweaking to the rest of the diagram.

MermaidJS supports flowcharts, sequence diagrams, class diagrams, state diagrams, ERD (entity-relationship diagrams), Gantt charts, pie charts, requirement diagrams and User Journey diagrams.

On my last post I actually used it to show the sequence of flows between WordPress, WooCommerce and webhooks.

WordPress Plugins

Working at Automattic, all our internal documentation and institutional knowledge is recorded using WordPress (you should really checkout using p2’s — it’s an awesome tool). Using Mermaid in this environment has been fantastic — Mermaid can be supported anywhere you have HTML (and I can be exported to SVG too), but as WordPress is running 40% of the web, this seems like a pretty good place to start.

In fact MermaidJS has a page dedicated to integrations of their software and mention a couple plugins:

  • WordPress Markdown Editor (seems to have been rebranded to WP Githuber-MD — see below)
  • WP-ReliableMD: The page was in a different language than I understand and I made a snap decision to avoid it 😒. Should I revisit this decision?

Here is my un-asked for 2Β’ about some different WordPress plugins supporting MermaidJS.

WP Mermaid (and WP Githuber-MD)

These plugins are both created & maintained by Terry Lin. I think they’re both great. They support Gutenberg blocks. They display the Mermaid text. Give them a shot and see if they float your boat.

Personally, I didn’t need the full markdown editor, and WP Mermaid had a couple small annoyances that I couldn’t solve. I submitted a couple PR’s, but decided I could use something a little simpler, which brings me to my own plugin…

MerPress

I can’t take full credit for this plugin. My coworker Michael Pretty actually did most of the work on this iteration and I submitted it to WordPress.org. I’m giving him full-credit. He did an awesome job with it.

A couple features I really like:

  • MerPress keeps the wptexturize functionality. Posts preserve the capability to convert simple characters like quotes and dashes into the nice curly quotes and dashes (e.g. “”, –, —).
  • In the Gutenberg editor, the block is encased in a code block letting you edit the code with a monotype font.

Some features I would like to see:

  • Ability to change the theme site-wide and per block
  • Add other configuration parameters to the block
  • Support 3rd party CDN (something that WP Mermaid already does)

Conclusion

If you learned nothing else, you now know I need pictures in order to comprehend complex concepts. Hopefully it’s also obvious that Mermaid is a neat tool and can be used to quickly share intention and purpose and thoughts without all the words.

graph TD
  ad(all done) --> te(the end)

Please let me know if I’m missing a plugin that I should try or if there is something you would like to see added to MerPress…

Categories
Software Development

Working with WooCommerce Webhooks

I recently went deep into the WooCommerce Webhooks code. I’m preserving my thoughts here (if anyone needs more help — feel free to comment — I’m happy to share what I know).

Webhooks?

TL;DR; A separate website can be notified when you make a sale by creating a webhook.

WooCommerce provides a method to send announcements of actions that occur on your site to any URL. These announcements (topics) consist of WooCommerce domain objects (orders, products, subscriptions, etc. also generically named resources) and include their life-cycle events (creation, updates, deletion, etc.). The WooCommerce documentation has a pretty good writeup about this.

WooCommerce Webhooks Handling Overview

sequenceDiagram

participant WordPress
activate WordPress
WordPress ->> WooCommerce: load woocommerce
activate WooCommerce
WooCommerce ->> Webhooks: instantiate webhooks
activate Webhooks
loop Every webhook
Webhooks ->> Webhooks: hook topics
Note right of Webhooks: An topic is defined by a <br> resource and an event.<br>-<br>A topic has a mapping to one or <br>more WordPress hooks. When the <br>hook fires (i.e. do_action),<br> the webhook is enqueued.<br>-<br>See $webhook->enqueue()
end
deactivate Webhooks
deactivate WooCommerce
WordPress ->> WordPress: ...
WordPress ->> Webhooks: Webhook Event (e.g. Order created)
Note right of Webhooks: Events are added to a global<br>array with: 1️⃣ the Webhook object<br>and 2️⃣ the args passed to the hook.<br>-<br>See $webhook->process()
WordPress ->> WordPress: ...
Note right of WordPress: shutdown
loop Enqueued Webhooks
WordPress ->> Webhooks: deliver
Note right of Webhooks: Actually 2 different methods to <br>deliver. Defaults to async (using<br> Action Scheduler), but can<br>run synchronously.<br>-<br>See $webhook->deliver()
end
deactivate WordPress

Using Webhooks For Fun and Profit

Topics aren’t limited to just WooCommerce domain objects. You can use any WordPress hook to send a webhook. Using the resource “action”, add the hook name as the event and you will receive the first argument passed to the hook as the parameter in your webhook. Events must start with wc_ or woocommerce_.

Example Payload

{
  "action":"woocommerce_my_custom_event",
  "arg":{
     "test":"something I passed in"
  }
}

Deliver custom topics

Using the action resource is limiting.

  • The hook name needs to have a special prefix
  • The shape of the data returned will be wrapped in the arg parameter.
  • WooCommerce uses ActionScheduler for storing/managing async events, capping the size of the data at 191 characters. (Async handling can be disabled to avoid this limitation).

What I really want is my own resources/events. For this example let’s pretend we needed to add our own bookmark object, and we want to send a webhook whenever a bookmark is created.

First, add our topic to the list so it can be appropriately enqueued. This mapping is from topic β†’ hook. When the hook is called (e.g. do_action( 'send_bookmark_created_webhook', $bookmark->id ); ) The webhook is added to the queue.

/**
 * Add webhook topics.
 *
 * This is used in the startup portion of webhooks, and is called for each webhook.  As the webhook has the reference
 * to the specific topic, just return the WordPress action to listen for this topic.
 *
 * @param array $topic_hooks
 * @param \WC_Webhook $webhook
 */
function add_topics( $topic_hooks, $webhook ) {
	$topic_hooks['bookmark.created'] = 'send_bookmark_created_webhook'; // When we create a bookmark, we'll fire this action.
	return $topic_hooks;
}

add_filter( 'woocommerce_webhook_topic_hooks', 'add_topics', 10, 2 );

Now we need to create our payload (avoiding the 191 character limit).

/**
 * Build a payload for our custom topics.
 *
 * @param $filtered_payload mixed  Passed from the existing build_payload function.
 * @param $webhook_resource string Resource parsed from the webhook topic.
 * @param $arg mixed               Args passed to the trigger for the topic.
 * @param $webhook_id int          Can use this to rehydrate a copy of the webhook (e.g. new WC_Webhook( $id )).
 *
 * @return mixed
 */
function build_payload( $filtered_payload, $webhook_resource, $arg, $webhook_id ) {
	if ( 'bookmark' === $webhook_resource ) {
		return get_bookmark( $arg );
	}
	return $filtered_payload;
}

add_filter( 'woocommerce_webhook_payload', 'build_payload', 20, 4 );

Finally, whitelist the webhook.

/**
 * Before the webhook is sent this hook determines if it should be delivered.
 *
 * Have to override this because the custom types.  Alternately could continually overwrite
 * `woocommerce_valid_webhook_resources' and 'woocommerce_valid_webhook_events` like is done
 * in the store currently.
 *
 * @param $should_deliver bool Existing value
 * @param $webhook \WC_Webhook
 * @param $arg mixed           Delivery body
 */
function should_deliver( $should_deliver, $webhook, $arg ) {
	if ( 'bookmark.created' === $webhook->get_topic() ) {
		return true;
	}
	return $should_deliver;
}

add_filter( 'woocommerce_webhook_should_deliver', 'should_deliver', 10, 3 );

At the end of the day

If this helped you or if something remains confusing let me know, I’m interested in hearing people’s pain points and wish list items. Personally, I’m looking at adding retry functionality to this infrastructure and will add another post when I get that sorted.