What is a queue and why are they useful

PHP is single threaded, so concurrency is managed in the software layer. Using Laravel’s queue, developers have a first class abstraction for asynchronus behavior. The queue helps the running application to prioritize and execute code asynchronously. One of the nice things about Laravel’s implementation of the queue is the ability to refactor a performance bottle neck into a queued job.

Imagine your working on a social media application which has Communities, and Users. Users can add Conversations to a Community. Users can then add a Comment to a Conversation or another Comment by filling out a small form built with Livewire and Volt.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

...
new class extends Component {
    #[Validate('required')]
    public string $body;

    public Conversation $conversation;
    public $root;

    public function submit() 
    {
        $this->validate();
        Comment::create([
            'conversation_id' => $this->conversation->id,
            'user_id' => Auth::user()->id,
            'commentable_id' => $this->root->id,
            'commentable_type' => $this->root::class,
            'body' => $this->body,
        ]);
        $this->dispatch('comment-created', rootId: $this->root->id);
    }
}; ?>

<form wire:submit="submit" class="space-y-6">
    <div>
        <x-input-label>Add Comment</x-input-label>
        <x-text-input class="block w-full" wire:model="body" />
        <x-input-error :messages="$errors->get('body')"/>
    </div>
    <x-primary-button>Submit</x-primary-button>
</form>

Let’s say this results in a performance bottleneck in our application. We can extract the behavior from the front end and use a queued job triggered from the frontend, and allow the application to insert the new comment asynchronously.

Since a comment can be left on a conversation or a comment, I’m using Laravel’s polymorphic relationships to pass the $root of the comment into the constructor.

Jobs can be created using an artisan command, which I recommend: php artisan make:job. You can pass a name in, or artisan will ask you for one. In this case, I named it InsertComment, and extracted the behavior from the ui component.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

...
class InsertComment implements ShouldQueue
{
    use Queueable;

    /**
     * Create a new job instance.
     */
    public function __construct(
        public Conversation $conversation,
        public User $user,
        public Model $root,
        public string $body,
    )
    {}

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        Comment::create([
            'conversation_id' => $this->conversation->id,
            'user_id' => $this->user->id,
            'commentable_id' => $this->root->id,
            'commentable_type' => $this->root::class,
            'body' => $this->body,
        ]);
    }
}

…and then we can update our form component to use the new Job by firing the ::dispatch() static method, passing in the parameters for the constructor. The comment root is enumerated into it’s id and class, so it doesn’t need to be typed specifically, but this gives future developers some guide as to what a “root” may be, in context.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

...
new class extends Component {
    #[Validate('required')]
    public string $body;

    public Conversation $conversation;
    public $root;

    public function submit() 
    {
        $this->validate();
        InsertComment::dispatch(
            $this->conversation,
            Auth::user(),
            $this->root,
            $this->body,
        );
        // $this->dispatch('comment-created', rootId: $this->root->id);
    }
}; ?>

<form wire:submit="submit" class="space-y-6">
    <div>
        <x-input-label>Add Comment</x-input-label>
        <x-text-input class="block w-full" wire:model="body" />
        <x-input-error :messages="$errors->get('body')"/>
    </div>
    <x-primary-button>Submit</x-primary-button>
</form>

As you can see, I’ve commented out the code that would previously dispatch a browser event after a comment was created. In the old code, that was fine. Single-threaded-PHP would interpret the code line-by-line and by the time the browser event was firing, we could be sure a new comment existed in the DB, but now we cannot be so sure. All things considered, the application may prioritize the generation of the browser event over the insertion of the comment. So the next step is to broadcast the event to our clients. While it might seem like the job is a perfectly good place to broadcast from, one of the other superpowers of jobs on the queue is they are isolated to maximize performance. The end result is that any data that you didn’t inject into the job’s constructor will not be available. The queue has access to the application in memory, but likely won’t be able to track any changes in the app state.

Broadcasting server events to a Livewire component

Laravel’s broadcasting capabilities, as of Laravel 11, are no longer configured by default. Websockets and RTC are managed by a separate server alongside your web server, and that server can either be managed by you or you can use a 3rd party provider to serve your websocket connections. If you already have websockets and broadcasting configured on your app, you can move on, but if not you may want to review Laravel’s docs on broadcasting. If you are starting from scratch, I would recommend using Laravel Reverb.

Broadcasting is related to the queue, as the queue is tasked with prioritizing the events you are broadcasting, but a broadcast event can be received by a user client, and then the user can request a fresh set of data and a new UI, which is how they differ from a job on the queue. Similar to a job, an event can be set up using an artisan command: php artisan make:event.

After creating the event, I set it up as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

...
class CommentCreated implements ShouldBroadcast
{
    use Dispatchable, InteractsWithSockets, SerializesModels;

    /**
     * Create a new event instance.
     */
    public function __construct(
        public string $rootId,
    )
    {}

    /**
     * Get the channels the event should broadcast on.
     *
     * @return \Illuminate\Broadcasting\Channel
     */
    public function broadcastOn(): Channel
    {
        return 
            new Channel('comment.' . $this->rootId);
    }
}

I made two changes to the class as created by artisan.

1. I had the class implement ShouldBroadcast
2. I changed the broadcastOn method to return a single channel instead of an array of channels, with the default being private.

For a chat or other peer-to-peer style socket communication, you will want to make sure that the correct client is being communicated with and that they are authorized to be on the channel, but for updating a web page after an insert, using a public channel allows comments created by one user to show up in real-ish time for a second user viewing the same page.

I then modified the handle method on the job to queue the event after the comment is inserted:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

...
use App\Events\CommentCreated;
...
class InsertComment implements ShouldQueue
{
    use Queueable;

    /**
     * Create a new job instance.
     */
    public function __construct(
        public Conversation $conversation,
        public User $user,
        public Model $root,
        public string $body,
    )
    {}

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        Comment::create([
            'conversation_id' => $this->conversation->id,
            'user_id' => $this->user->id,
            'commentable_id' => $this->root->id,
            'commentable_type' => $this->root::class,
            'body' => $this->body,
        ]);
        CommentCreated::dispatch($this->root->id);
    }
}

Your app needs routes set up for broadcasting, so when you configure your app for broadcasting, a channels.php file is added to your routes directory.

1
2
3
4
5
6
7
8
9

use Illuminate\Support\Facades\Broadcast;

Broadcast::channel('App.Models.User.{id}', function ($user, $id) {
    return (int) $user->id === (int) $id;
});

Broadcast::channel('comment.{rootId}', function ($rootId) {
    return true;
});

The route I added gets a route name as the first parameter. This will be used later to identify which element may need to be updated. The method passed in as the second parameter should return true or false to determine if the client is authorized. Similar to a job, any data required to do this would have to be passed in, because extracting data from the application in memory is not trustworthy “on the queue”. Now we can go back and modify our Livewire components to detect the event and update the component. Here’s the view that was previously detecting the browser event to display a new comment. The form we looked at previously is pulled in on line 49. The method to handle the event is on line 7.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

...
new #[Layout('layouts.app')] class extends Component {
    public Conversation $conversation;
    public bool $showRootComment = false;

    #[On('comment-created')]
    public function commentCreated($rootId) {
        if ($this->conversation->id === $rootId) {
            $this->reset(['showRootComment']);
            $this->conversation->refresh();
        }
    }

    public function mount(Community $community, Contribution $contribution) {
        $this->conversation = Conversation::where('community_id', '=', $community->id)
                                          ->where('contribution_id', '=', $contribution->id)
                                          ->sole();
    }

    public function with() {
        return [
            'comments' => $this->conversation->comments()->paginate(10),
        ];
    }

}; ?>

<div>
    <x-header>
        <div class="flex items-center justify-between">
            <x-h1>
                {{ $conversation->contribution->name }}
            </x-h1>
            <div class="flex flex-col space-y-1">
                @if($conversation->community->userIsSubscribed(Auth::user()))
                <x-primary-button wire:click="$toggle('showRootComment')">
                    Comment
                </x-primary-button>
                <livewire:components.turn.toggle :root="$conversation" key="vote-{{$conversation->id}}" />
                @endif
                <x-secondary-button-link :href="$conversation->contribution->url" target="_BLANK">
                    Visit
                </x-secondary-button-link>
            </div>
        </div>
    </x-header>
    @if($showRootComment)
    <x-content-card>
        <livewire:components.comment.form :conversation="$conversation" :root="$conversation" :key="$conversation->id" />
    </x-content-card>
    @endif
    @if($comments->count())
    <x-content-card>
        <x-h2>The Conversation</x-h2>
        <div class="space-y-6">
            @foreach($comments as $comment)
                <livewire:components.comment.card 
                    :comment="$comment"
                    :conversation="$conversation"
                    :root="$comment"
                    :key="$comment->id"/>
            
            @endforeach
        </div>
        {{ $comments->links() }}
    </x-content-card>
    @endif
</div>

Under the hood of Reverb, Laravel is using a package called echo to broadcast to the client browsers. Livewire can detect these events and react to them just like any other browser event, so you can modify the above code as follows to have it track the browser event that is generated when the server event is broadcast:

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

...
new #[Layout('layouts.app')] class extends Component {
    public Conversation $conversation;
    public bool $showRootComment = false;

    public function getListeners() {
        return [
            "echo:comment.{$this->conversation->id},CommentCreated" => 'commentCreated'
        ];
    }

    public function commentCreated() {
        $this->reset(['showRootComment']);
        $this->conversation->refresh();
    }
...

So this closes the loop: we queued a job, triggered by a user, when the job finished, it queued an event. The event was broadcast active sessions, and the component refreshes it’s state, although to call it completely finished you will probably want to show some kind of loading state in your UI when kicking off the job that can be cleared when the event fires and is broadcast.

I have spent the last four years of my life as a professional PHP developer working on a Laravel web app for a final mile logistics company. So a lot of what I did was to build specific web pages to support normal business operations like creating, retrieving, updating, and deleting orders, managing customers, managing user rights and permissions, and generating invoices and payments.

For the most part, our database schema was where complexity was housed, and our code was about as transparent as it could be using PHP. My mentor, also my boss, had a lot of experience working on a large eCommerce web app built in Ruby-on-Rails, which was at one time the new and hot framework in web development. As we worked our way up from Laravel 5 to 7 and eventually 10, we started implementing some of the more modern syntax and niceties from these versions. The log-jam started to clear and features really started to fly! So obviously the company downsized it’s development team by half and the last bit of code in my portfolio happens to be from when I was in boot camp, so here we go lets build a Reddit clone with Laravel and talk about how it’s really slick now.

Without burying the lead, I’ve got a good jump on this project, and I’ve got a few things to say about Laravel 11, Livewire and Volt.

First, Volt: If you look in the controllers directory, you’ll see there are none. They are not necessary when you’re using a Volt route and a full page Volt component. Looking more closely, you can see that route model binding works the same way it does with existing Laravel routes. The resulting models can be type-hinted in the mount method on the Volt component. One of the really nice improvements added by the Volt package is the ability to have the logic and markup in the same file.

While a lot of the credit I’m giving here is handed out to the new bits that have become easier to implement in later versions of Laravel, the use of Blade syntax and modular Blade components is really the star that allows the Volt components to be oh-so-readable. I assume had I written out all the Tailwind classes in the component I linked above, it would probably be a nightmare to look at. While I’ll never be a designer, my time at a marketing firm did help me establish a few good habits when it comes to separation of code from markup, the separation of markup from styling, and how to build reusable UI atoms.

I’ve pushed fewer than 100 commits to this project as of right now, but I have accomplished the following:

• Implement boilerplate Auth
• Create basic social media models:
  • Conversation: a user creates these by submitting a URL to a community
  • Community: a repository for conversations, users can subscribe by ‘claiming’ the community.
  • Users can comment in a conversation.
  • Users can comment on comments infinitely.
  • Users can vote on comments and conversations.
• Server side jobs: voting is queued
• WebRTC / websockets: new votes are broadcast and update when a voting job is completed

As always, testing in Laravel is incredibly easy. Without controllers, it seems much easier to figure out what and how to test my models. Coverage is currently around 85%. The issue in coverage emanates from the parts of the Auth system I’m not using. Livewire really does allow you to think of your web application as a set of views that are intrinsically bound to the models they are hydrated by.

While I’m sure Alpine.js is a fine lightweight JS library, but here’s what I love about modern Laravel:

• Don’t write controllers
• Don’t write JavaScript
• Bring unit-testable model behaviors directly into the view

LLMs will eventually, some day in the future, be as smart as humanity was at some point in the past.

As of the summer of 2024, the pace of Large Language Model development has slowed. The concept of the eventual emergence of artificial general intelligence is waning in both its scale and popularity. Which means that it’s probably just about time for AI’s “killer app” to emerge.

LLMs are mostly a tool, by now. People smarter than I are starting to apply these tools to actual problems they might be good for, so for the rest of us, here’s a quick break-down of how these things work.

LLMs are born when fed a huge set of text data. For the most part, the data sets are big outdated portions of the internet with some of the garbage and useless nonsense scooped out. The “black box” is the analysis of the text which figures out the mathematical relationships between the words.

Afterwards, the models are “fine tuned” using sets of inputs in the form of a question and a known good answer. The “black box” is then told to use those question-answer pairs to refine its output to create answers that align with the known good questions and answers.

If the question is about something that doesn’t exist in the initial data sets, most of the big LLMs are able to acquire new text data. For example, if ChatGPT doesn’t know what you’re talking about, it’s allowed to use Bing Search to find new documents on the internet to ingest. This is called “Retrieval Augmented Generation” or RAG.

So that’s why I’m not scared, because LLMs will never (seemingly ever) know anything that someone hasn’t said before. LLMs will eventually, some day in the future, be as smart as humanity was at some point in the past. Even if LLMs are ingesting text generated by LLMs, if you dig deeply enough into the trail of documents parsed by any given LLM eventually you’ll find my angsty LiveJournal posts from 2001, or an essay about Roy Orbison completely wrapped in cling film.

And for what it’s worth, it’s only gonna take a few more of these before the industry backs away from these tools significantly.

So I listened to Wall of Eyes by Radiohead. I guess it was some kind of trick to confuse the press? They released it under some other name. Anyway, it’s a journey of how one would write a song if you had Thom Yorke’s voice at your disposal. It was really good, and it ended with a lilting Thom Yorke piano ballad. No notes, Radiohead.

J Mascis new album What Do We Do Now was very comfortable for a sometimes Dinosaur Jr lover. It had what you’ve come to appreciate from J’s solo work: dynamics. It lacks the obfuscation and complexity of a Dinosaur Jr. album, but it was very easy to listen to.

Every week for the past several years I have listened to a new album. To me ‘new’ means an album which was released within 7 days of me listening to it. I have learned about a lot of stuff that I really like by doing this. I assure you that the practice started in a very grass-roots fashion. Over time, it became a challenge to myself to see how long I could keep it going on, and now it’s just something that I do. Things like this have no right to get boring, though. So I’m going to try and take some time to document some of my experiences this year.

Zwierzakom by Cisnienie is an epic post rock experience. These extra long textured minimalist songs are very long. The tracks often build from a pulse and crescendo in the way that post rock does. The first tune is 20 minutes, and at the end the apparent crowd noise informs us that this is also a bunch of individual live takes. It’s impressive, but I was gasping for air by the fourth tune…which is 55 minutes in. Released 30 Dec 2023. Listened 3 Jan 2024

Letter to Self by SPRINTS gets the replay award for this post. This album has something that I can’t put my finger on. The singer is super emotive and seems to push hard on all the lyrics in that typical aggressive Irish way that seems to be very endearing to us Yanks. It’s like if Lemmy came back from the dead and the first thing he said was “I wanna hear something with guitars to make everyone forget Sleigh Bells ever existed” and SPRINTS was right there like “We’re doing it.” Released 5 Jan 2024, listened 8 Jan 2024.

I signed up for ChatGPT and had the following converstaion:

Alex Szeliga: Why does the porridge bird lay it’s egg in the air?

ChatGPT: I’m sorry, but the porridge bird is a fictional creature that exists only in a nursery rhyme. There is no scientific explanation for why it lays its egg in the air, as it is purely a fantastical concept created for the purpose of entertainment or education. The rhyme may have been designed to help children develop their imagination or language skills.

Alex Szeliga: Why are you sorry?

ChatGPT: As an AI language model, I don’t have emotions, so I don’t actually feel sorry. However, I used that phrase as a way to acknowledge that I was unable to answer your initial question about the porridge bird laying its egg in the air. My apologies if my previous response was unclear.

Alex Szeliga: Are you familiar with The Firesign Theater?

ChatGPT: Yes, I’m familiar with The Firesign Theatre. It was an American comedy group that was popular in the late 1960s and early 1970s. The group consisted of four members: Phil Austin, Peter Bergman, David Ossman, and Philip Proctor. They were known for their innovative and surreal comedy style, which often involved intricate wordplay, social commentary, and satire. Some of their most well-known works include the albums “Don’t Crush That Dwarf, Hand Me the Pliers” and “I Think We’re All Bozos on This Bus,” which are considered classics of the genre.

Alex Szeliga: They wrote a sketch called “The Breaking of the President” where a natural language computing model was “broken” by asking it about the porridge bird laying it’s egg in the air. In the skit, the language model responded with a lot of gibberish, but eventually settled on one answer: No.

ChatGPT: That’s an interesting piece of trivia! It’s not surprising that a natural language computing model from that time period would struggle to understand and respond coherently to a question about a fictitious creature like the porridge bird laying an egg in the air. However, it’s worth noting that with advancements in natural language processing and machine learning techniques in recent years, modern language models are now able to understand and generate more complex language and even generate creative and imaginative responses.

…Did ChatGPT just jive me?

“…because if we’re lucky tomorrow, we won’t have to answer questions like yours anymore”

I was raised in a family of musicians and music aficionados. No instrument or music was out of bounds, but no one was allowed to like music just to like it. The lowest form of music was music that was meant to be ignored like in an elevator or the grocery store. Our troupe were bards, not scholars, so extra emphasis was laid on untrained and naturally gifted players.

As a boy with a drumset in the basement, but not yet any actual drummers in the family, I took close notice to the few drummers mentioned in this way: Ringo, Levon, but especially Ginger. Two refrains: He was a jazz musician in London before becoming a global mega-star of rock drumming, and “his drum solos were so precisely tailored to the songs that you could hum the tune and the band would just pick it back up; It was an actual drum solo in the composition, not just a drummer flailing about wildly.”

As one does, I trusted this as gospel. It came directly from the people around me I trusted with my development as an artist. Then I started to learn about Mr. Baker. I watched the solos and the tape available. I listened to the recordings, learned about the man, his legacy and thought to myself, what is different about Ginger Baker and John Bonham? Why is it that more people will remember White Room than will know who John Bonham is, but more drummers will remember John Bonham than Ginger Baker?

Fucking ego. Both men had terrible ego problems, and their unwillingness to put it down behind the kit is what made them interesting to watch, but we’re done with watching our artists die before us for pay. We want them to live, and that has created JD Beck.

JD Beck is an American drummer, producer and songwriter. If you don’t know, listen to this then this, then you’ll know. They are currently promoting their first album and I’m very excited to watch their career and see what kind of new and interesting things they bring to the drums. Anderson .Paak; you’re on notice. Don’t let this one do anything dumb. I wanna see what Jaco would be like at 80.

JD is special because the way that JD plays the drums is respectful of how humans experience the drums. As much as Ginger and Bonzo and the like would have you believe that you need resonance and a broad range of sonic options to make the drums fit into a broader musical composition, JD has discovered that it’s not true. In much the same way that Jaco Pastoreus discovered a relationship with how people want to experience his instrument in their music, JD has taken an example from a generation of musicians who are digital natives. On top of all of that, the average quality of speakers have never been shittier. While Ginger and Bonzo will tell you 4 toms minimum, the average person can’t tell the difference between a drum set with 2 and a drum set with 5. While this is fairly unscientific, my wife can’t actually hear the chord changes in most songs with distorted guitars, but she’s not alone. While musicians and auteurs crave fidelity and the extended information it provides, the amount of auditory information coming off a drum-set is a lot. Need more proof? Turn on the radio to a station that plays pop music. Listen for a few minutes, then SING the drum part. Regardless of how you go about it or what song you hear, it will sound something like this. This is how tabla drum music is shared between players. Even a simple clay drum like the tabla, in context, presents so much information to the listener that the actual performance is obscured, so the players use a simplified language of mouth sounds that sound like drums. Mouth drums. The mouth drum set is always rudimentary. There are no extra tom-tom drums or cymbals on the mouth drum set. Similarly, there are no extra drums or cymbals on JD Beck’s drum set. The drums sound short and the cymbals sound high and scratchy. The floor tom and the bass drum make essentially the same noise, but the listener pays no penalty. In fact, the listener isn’t forced to think about that decision at all.

One of the things that makes JD Beck’s actual playing so special is that when they perform, JD makes the crowd feel the beauty of metric modulation, rudimental drumming, jazz comping, and a couple hundred years of tradition while completely missing the off-ramp to the drummer’s circle-jerk. By doing so, JD has created a path to feeling the way it feels to play the drums from your soul. John Bonham just wanted to share that with his family, and this profession tore him away from that. Ginger Baker is a sociopath who wanted to be a famous drummer. Of course they both loved the off-ramp to the drummer’s circle jerk. To be fair, JD Beck had a whole generation of hip-hop, synthpop, and electronic music producers who had already, by necessity, stripped most of the information out of the drumset, JD is just the bolded and brightest of a generation of drummers who will likely do the same.

Here’s what I mean, when I say JD is curating a selection of complicated musical conversations and making you feel them. In this exercise, you’ll listen to a short part of a song twice. The first time, you just listen to it. Just enjoy it, but bob your head or tap your foot.

JD Beck @ Zildjian Live: give it about 30 seconds to a minute.

The second exercise is to try and do that thing we do with all pop music: try and divide it into 4. Start back around @4:20 in the song, and whenever you feel comfortable start counting out 1, 2, 3, 4. You will immediately get that “rubbing my belly and patting my head” feeling. The music follows the familiar “4/4” pattern. You can stop and go back to listening for entertainment now. The tune is on their new album as a track called 2 SHRiMPZ featuring Mac Demarco.

Anyway, thank you for coming to my TED Talk. Musicians who play drums need to be protected, and not because they are in danger of becoming machines, but because they are in danger of being just a drummer.