Swift Package Indexing

Well, Happy New Year Sven.

Happy New Year.

Did you have a good break?

Oh, absolutely. Yeah, good break. Very relaxed. Not quite as cold as in Germany or in other

places. I've seen it's been really cold in Sweden. There's someone I'm following on

Mastodon and he's been subjected to minus 40 degrees Celsius.

Okay, that's a lot colder than it is here too.

Yeah, no, I can't stand the cold and just reading that gave me the shivers.

It's not very warm here. It's going to hit two degrees above zero on Friday this week,

but it's certainly not minus 40.

Yeah, no, that's just way too cold.

Did you have a good break as well?

Yes, I did. Thank you. Nice to see family, friends,

spend some time away from a computer a little bit, which was always nice to do, but ready

to get back to it and raring to go for 2024.

Excellent.

And maybe we should start there actually. So we should maybe talk about

some of the things that we hope to get to in 2024 with the package index.

Yeah, that sounds like a plan. We used to post

like things way back and we held that up for a couple of quarters, but then due to the way we

organise stuff that sort of didn't, it wasn't really feasible anymore. It didn't make much

sense anymore, but I think on a yearly basis, it's a nice thing to maybe look forward what

we're planning to do.

Yes. And of course, all of the things we're about to talk to today can change.

And will.

And will. Exactly. Yeah, exactly.

So, yeah, I think just in terms of I briefly thought about what's the rough plan. And I think

one of the things that I'd like to continue working on, and we started doing that a bit is

dependencies. We'll also talk a bit about that later on in the news section. We've started

tracking dependencies quite early on by just capturing the resolve dependencies,

which is the full dependency tree you get in a Swift package after package resolution. So that's

the transient dependencies included and the test target dependencies. That's literally everything

your package touches. And that's been in our database for quite a while,

but that's also not the best thing to report really for people who are inspecting a package.

You don't necessarily are interested in test dependencies or all the dependencies. You sort

of maybe want the first and direct dependencies and that sort of thing. We're starting to pick up

and we have those in the database as well now, the direct dependencies. So we can report in a bit

more detail what we've got, but there's a lot more work to do here to also prune up the test

dependencies and all that. And I think that's going to be really exciting because it's one of

the things that will give us a bigger ecosystem wide overview, just to the things you can do then

by looking at what packages are sort of central. They have lots of dependencies, packages that

depend on them. Is that dependency? Yeah, that's dependence. Now that's dependence or whatever,

dependence. That's the one. I'm kind of imagining we'd be able to maybe draw some graphs or

something around like these webs and see packages that are very cool. And we've talked in the past

about when we talked about the NPM ecosystem and the crates, the Rust ecosystem. There's some

statistic and analysis that they've done in their ecosystems that would be interesting to do in

Swift, in the Swift ecosystem. I'd really love to do that, try and replicate that, see if we see

some of the same problems there with the circular dependency graphs, which were kind of weird.

I wonder if there's something like that in Swift as well. All that sort of stuff falls

under that umbrella that I'd really love to focus on in the coming year.

And I know you've started doing a little bit of this already, not so much on the dependencies

display in the package index, but so building on some of the work that we already have in the

project for getting that list of complete, that total list of every type of dependency that we

currently have. There was a nightly process, or there still is a nightly process that runs

through GitHub actions, which goes and tries to find any dependencies that are dependent of,

sorry, that depend, no. I got it wrong. It finds any packages that are not in the package index,

which are depended on by other packages. And that process used to take about eight or nine

hours to run something like that every night. Yeah, yeah, exactly. It had increased to,

I think, nine and a half hours now, even because we've grown since we started doing this.

And yeah, this is a long one. And I know that we've, well, you just put some work into,

and we've just put into production a version of that script that now uses the dependency work

that we do in the main package index to shortcut all of those searching for dependency steps and

just read the dependencies out of our application. And that took the job from nine and a half hours

down to certainly less than half an hour, right? Yeah, it's like five minutes or something like

that, because it's effectively, we were previously crawling every package, downloading the package

manifest and doing a package resolution to get the package resolved out of it, which is something we

already do continuously in the Swift package index. So we have those things in a database,

and this was effectively just ripping that out, asking the database for everything in one go.

And that query is even faster. I mean, that query literally runs in 50, 100 milliseconds,

and it's the downloading of the data that maybe takes maybe half a second. And the remaining time

is then to validate these packages. But we have a very fast way now to find out which packages

are being referenced but aren't in the index yet. And that was really, really helpful to get that

nightly process under control. It also helped us find packages that we didn't actually find with

the other process in the past, because the way the other process worked was to download the

manifest of the main branch, of the latest version of the main branch. But what happens then is that

we have packages in the index that are old, that don't have new revisions, and they never moved

forward. And they were actually referencing, they had all the manifests that were referencing

package versions that weren't, that were only live at the time, but aren't live anymore.

This sounds probably way more complicated, and it's hard to follow. But the thing is,

we're now unearthing packages that are being referenced that we didn't actually see before.

So if you are following our Swift package updates, Mastodon account, you will see loads

of new packages appearing. And those are the packages we're now finding and adding to the

index because we've made that change. And we've also changed, we're now accepting forks as that

people are depending on, and adding them to the index, because they are packages like any other

package, and they're being used. So they should actually be in the index, so we can properly

account for all the dependencies we've got and have them in the index.

And this is the point I was going to make, and it's the influx of these additional dependency

packages has really tripped me up this week and made... So when I'm preparing for this podcast,

I go through the list of new packages to see what's interesting and what's been added.

And with all these, quite a significant number of packages, more than 100 packages being added

by this script, I made that job much more difficult than usual, because it's a whole

load of forks and packages that I just depended on that have maybe been around for years and years

and years that we might not feature on a podcast like this, but as you say, should be in the index.

And one thing that, in the context of what are we going to work on this year,

one thing that this has really highlighted for me is that we quite desperately need a feature

which shows which packages are forks of other packages. And if we have the other package in

the index, then we should also refer to that package because it's quite... In fact, I was

tripped up myself by a fork of plot, John Sundell's plot library, which is an HTML generation library,

which we use in the package index. And I was going through the list of packages and I saw plot go up

and I thought, "That's strange. We've definitely got plot." And I opened it in my browser and

looked at it and it took me a second to realize that it was my fork of plot that...

Was it your own fork? Yeah. I saw that flow come through.

And that deserves to be in the index, but it's also... It should be really obvious. Yeah,

it should be really obvious that that is a fork and not a brand new project.

Well, if any fork deserves to be in it, it's your own, right?

Well, it takes my total number of packages in the index up to two. We've got left pad,

which is the test package that we have. And now a fork of somebody else's work.

Nice. So yeah, that feature, we just need to

add a little bit of extra metadata to those fork packages. So just be careful if you see

new packages that you think should already be in the index, it may be a fork right now,

because we're not very clearly displaying that. There's a couple of things that I'm

interested in working on in terms of broad kind of themes for this year.

One is I'd love to do some more work on the package score feature. So we've talked about

this in previous episodes of the podcast and in blog posts recently as well. We have a

internal, in fact, not just internal anymore, it's now publicly available package score. And

you can see the score for any package that's in the index. But the list of kind of factors that we

use to score each package could be improved. And there's a thread open in the discussions

forum on the package index GitHub project, which we'll put a link to in the show notes.

If you have opinions on that, there's already a discussion going there.

I would like to do some more work on that this year, and kind of bulk up the significance of,

not the significance, because it won't be any more significant than it is at the moment,

but bulk up the kind of the evaluation of packages to promote better packages in the

index. So that's one thing I want to work on this year. And I think the other is something we've

been kind of discussing a little bit is over time, I don't know whether you remember, actually,

I know you will Sven, but I don't know whether anyone else remembers that when

the package index first started, it was entirely a metadata, metadata only information on the

package page. So we didn't actually display the readme file until quite a while after we launched.

And instead, it was it was purely metadata, it was it was a lot of the metadata that you see today,

even build compatibility and those results on the package page, and then a link to

the GitHub repository. And of course, over time, we and in fact, it was never our intention not

to have the readme file on there, it was just a matter of, you know, getting things done.

And obviously, we did add the readme file. And here we are today. But over time, that metadata

section has grown and grown and the compatibility matrix takes up quite a lot of space. And it

pushes that readme too far down the page. So one of the things that we've been chatting about this

morning, actually, is whether we can look at that page design again, and maybe think about trying to

pull that readme file back up the page a little bit, we're not going to get rid of the metadata,

there's definitely going to be metadata at the top, because that metadata is really important.

But it would also be good to have at least the first few lines of the readme visible

on a reasonably sized screen, which is currently not the case.

Yeah, definitely. I think the there's ways to condense that down a bit, I think,

especially going forward, we might see fewer Swift versions that we need to display,

you know, that sort of stuff. And the compatibility matrix, as you mentioned,

might be might need to shrink down a bit anyway. It's we got all these Apple platforms, maybe

there's something we can do there. It's it's, it would be nice to pull up the readme a bit more.

Yeah, agreed.

But of course, I'm sure we'll do a whole load of other things as well.

Yeah. Just to mention, there's there's no year without new Swift versions. So that's

certainly something we'll be looking at trying to add those as early as possible.

Who knows when Swift 6 will come out if it's this year. Rest assured, we'll be looking at

compatibility info there because we we know that Swift 6 will have will be potentially

have sourcing compatibility. So it's it is possible to have syntax changes there.

That's allowed. So that's allowed. That's something that is that is in the cards,

I should say. And we'll definitely make sure that we test this early so people can

can maybe use us there as the if they haven't set their own CI up or or just want to rely

on another service to have a look there and give an overview. So we're definitely trying to be

prompt with testing for Swift 6 as much as possible.

So when we've been doing a little bit of planning already with this and talking about Swift 6 and

you know, there's a Swift 6 strict compatibility mode, which is the concurrency checking that's

coming in that's potentially source breaking in terms of like there were things that with Swift 5

it would let you do and in Swift 6 it will it will strictly stop you doing some of those things.

And there's two modes you can run the Swift 6 compiler in, which is with strict compatibility

testing for concurrency issues on, which I believe is going to be the default for Swift 6.

And then Swift 6 in Swift 5 mode, which is if you want to use the Swift 6 compiler,

but you still need Swift 5 concurrency checking, then you can switch the Swift 6 compiler into

that mode, which we're not going to test for explicitly. But what we will be doing

is testing the latest version of Swift 5 at that point as well. So you'll have Swift 6

compatibility with strict mode on and then 5.9, 5.10, whatever it is at the point that

Swift 6 comes out, we'll be testing the Swift 5 mode. Have I got that right?

Yeah, exactly. I think there's, we don't expect there to be a significant difference or any

difference at all between Swift in 5 mode versus the latest 5 version at that time. So that's why

we're not really going to, we're not planning to spend effort on running those two modes in parallel.

Exactly. So, but of course we don't know, I don't know whether the public, the schedule

for that has been announced yet, but I'm not aware of it. So it's something that we will do

as and when those versions come out. Yeah. And well, I think there's no new

Apple platforms to plan for this year, right? Vision OS just came out and is...

Well, the good news is we already support it. Yeah, we already do. And I'm not aware of any

rumors about new Apple hardware or OS releases that we might be planning for. So I think on

that front, we're well covered at least. There is no new Apple hardware until the

day that Tim Cook announces it, right? It doesn't exist until he steps on stage.

That's how that works. Yeah, exactly. Yeah. I think you're right.

I think Vision Pro and obviously the Vision OS SDK, which did get announced, I think yesterday,

the release date for that, which I think is February 2nd, maybe, that we already support

in our compatibility matrix. What we will do is we'll update the version of Xcode that we use to

do that compatibility checking to 15.2, which again, I believe was released yesterday,

but really no changes that we need to do there. Yeah, exactly. And my understanding is the

515.2 beta one, I believe that we were using already had the... My understanding is how the

Vision OS SDK version that is actually shipping. So there won't be like the compatibility right now

should be up to date already, even when we change to the newest 515.2 version, that should all

remain the same, fingers crossed. That's great.

There is another bit of news we may want to briefly talk about. And that's a really interesting

post I saw coming through this week on Mastodon. And that's a blog post about

when everything becomes too much by Frost Abukadi. Did you see that, Dave? Did you have

a chance to look through that post? I did. You sent me a link to this earlier

today. And I did. I did give it a read. Yeah. So this is really fascinating. And

what a mess. What a mess. Yeah. And this is not about pointing fingers or just, you know,

laughing about what's going on in NPM. This is just general ecosystem stuff. And NPM

is pretty much, you know, it's a huge ecosystem, like orders of magnitude bigger. And it's sort of

the things they hit, they are going down paths that the Swift ecosystem hasn't even begun walking

down yet. So there's interesting stuff there. You could say they're stress testing dependency

management. Pretty much, yes. Yes, they definitely are. And some of this is inherent in how

the ecosystem works. I think it is fair to say that node packages are smaller. You know,

they bundle up smaller chunks of functionality into smaller packages. And that by nature then

have way more dependencies and all that. And that has a knock-on effect to how that works. But

that is just one aspect of it. And the other is just more exposure to more people that try

things. And what this is about is that someone created an NPM package called everything,

which, well, depends on everything. They have effectively made this package depend on every

package in the NPM registry. And just think about what that means. If someone went on to do NPM

install of that package, they would effectively DDoS themselves because it would start downloading

every package. And this just sort of spiraled out of control because it compounded with another

problem that happened in 2016. And that was the LeftPad incident.

Ah, LeftPad gets a second mention of the episode.

Yeah, second mention. There you go. So in 2016, the NPM world broke because the author of this

LeftPad package removed it at the time. And this was actually a package that was used in lots and

lots and lots of other packages. And that led to problems because then they wouldn't build anymore.

Lots of CI systems, is my understanding, started breaking and failing because they couldn't resolve

that package at build time and all that. And as a consequence of that mess at the time,

they established a rule that you can't unpublish a package once another package depends on it.

And now you can imagine what happened once everything started depending on every package

in NPM. That meant that no package could actually be unpublished anymore. So it was

effectively blocking everyone from unpublishing their packages after this. And just imagining

about how, you know, one thing that was made or one rule that was implemented to fix one thing

can then completely backfire if another thing goes horribly wrong or, you know, another,

you could call it an abuse case happens and sort of messes everything up. I don't want to be in

the shoes of people trying to manage this whole mess, but it must have been really horrific.

I think they found a solution in the end by just pulling the whole org down of the people who

published this. The people themselves actually were apologetic when they realized what they've

done. They want to do the right thing and unpublish the package, but they couldn't

themselves anymore because they'd been blocked out of unpublishing by, I guess, someone using

their package or something like that. I don't remember what the actual effect there was,

but it was just a big mess, but quite interesting as a look ahead.

I think it said in the post that the solution was to make the repositories, the everything

repositories private. And I think that's the workaround that they got to, because you can't

stop somebody taking a repository private. So they must have had to deal with that situation.

And I think that's what let them unwind this, if I read the post correctly.

But I mean, I do feel for the person who did this, because I genuinely believe that it was

done as a little bit of a lighthearted joke, or certainly its intent was a lighthearted joke.

But what a mess. Yeah, I mean, it's, can you imagine sitting there and suddenly realizing,

you know, it's like when that production incident happens and you suddenly realize you are connected

to the wrong database when you drop the tables. That's the sort of feeling that must go through

you. I mean, unless you really do it with the intention of breaking stuff. But if you didn't,

and I don't think they really did, then that is rough.

And it's the kind of thing, I mean, certainly,

Swift is a very different dependency environment than NPM. And we don't have this kind of situation,

or even really the potential yet for this kind of situation. But I think it's good for us to be

aware of what's happening in these other dependency environments. And certainly this one,

when, as I read more and more of this post, I thought, "Oh, this is, oh yeah, this is really,

it's really, really bad." I would encourage you to read the post. It's really good. Yeah.

- Yeah, we'll add a link to the show notes for people to read up on this.

Shall we do some packages or...

- Yeah, we could do some packages. Actually, I have a bonus extra little package,

which is not really a package recommendation this time. Well, but it's not in my normal theme of not

recommending a package. There may be one of those coming up later though. This was something I

spotted as I was doing a troll through the RSS feed. And I really liked the marketing approach

that this package has, which is to have no readme file whatsoever. So there is no information about

what this package does in the readme file at all. The name of the package is VGSL and the package

description says everything you really need to know. It's a very good Swift library. So that's

all you need to know. You should import this package immediately because it's a very good Swift

library. So, but no, I do have three actual recommendations as well. And the first of them

is by Daniel Lyons and it's called Plus Night Mode. And one thing that I... So I bought a new

watch this year in September or October, whenever they came out. And I don't know whether it's

something specific to the new watch. I think it may be to do with a new watch face, but when the

ambient light in the room is dark, the face and everything on it turns red, which is a really nice

way to not strain your eyes in the evening, or if you wake up at night or something like that and

try and look at your watch. And it's a... So everything goes kind of monochrome, but not

black and white, red and black. And this package by Daniel is a Swift UI view modifier, which you

can add to the bottom of any view hierarchy. So your entire navigation stack, for example,

could have this view modifier on it called Observing Night Mode. And you can pass a switch,

a Boolean in there to turn night mode on or off, which is something you'd have to track yourself.

So this would be an app specific feature. And it basically turns every view in that view hierarchy

into a red and black monochrome view, no matter what the original appearance of it was. And I

think that's lovely. I think, first of all, there's a couple of nice things. I think it's a

great feature. But secondly, I love the kind of cascading nature of Swift UI view modifiers in

that if you add them to the top level, then you can just do something like this, which would be

very difficult to manage in UI kits. I mean, you could still do it in UI kit, but it would not be

as easy as adding one view modifier in a UI kit. So I like that. And I thought it was worth mentioning.

- Yeah, I love that about Swift UI as well. I think the redacted stuff works the same way,

right? Where you can stub in, I don't know how best to describe it, redacted UI elements,

like text is not displayed as text, but as placeholders, like rectangles and stuff. I think

that's really nice and really powerful that you can just tack this on and everything just changes

into a different mode. I think that's really fascinating and super useful.

- I hope apps take advantage of this because I'd love to see this feature in apps that I use.

- Yeah, I have the same. I love this about the watch where it turns red. It's really nice.

Right. My first pick is called package-benchmark by Joachim Masila.

And this is a great package when you're looking to benchmark your code.

What it effectively does, it gives you a function or instrumentation to run a closure that gets

benchmarked. And it supports many, many metrics like CPU time, both like actual CPU time or

work clock time, throughput, memory allocations, threads, like how many threads you're using,

disk stats, all sorts of things. There's an amazing range of metrics you can track.

And I've been following this package for quite a long time, but I've always delayed talking about

it because I wanted to really give it a try. And it was clear from looking at the readme that this

is super comprehensive approach to benchmarking, but yet you'd also need to invest a little time

to understand what it reports and why that is important. That sort of held me back until I

actually sat down and tried using it. And it's not because the package itself is making this

complicated. It's because the topic is actually complicated and more complicated than you might

think. And that these details aren't ones that you should just gloss over and just take an average or

minimum when you benchmark stuff, because that's the thing you sort of reach for. I certainly,

whenever I benchmarked something in the past, you run it a few times, you look at it, well,

there's not too much jitter. I'll take the best one. You never really know what you're doing.

Typically when you measure, you do it a few times, you maybe calculate an error rate based on the

standard deviation, that sort of stuff. The thing is, when you do this, there are certain assumptions

that go in when that is actually valid and when it isn't. And mostly, almost always when you're

measuring benchmarks, these assumptions are wrong because of the way the distributions are shaped,

you can't actually calculate a standard deviation. And it doesn't make sense. And the framework

really leans into that. So what it does when you run a benchmark, it gives you so-called percentiles.

So it gives you a range of percentiles and measurements for those. So what that means,

a percentile is, let's say, percentile 50 means you're halfway through your distribution.

Everything is at least as fast as this and not slower. And the higher percentiles, for instance,

P100, 100th percentile means this is the slowest result you got. And the first one, which is called

P0, is the fastest result that you've got. So effectively, what this does, it gives you

certain points along the curve of your distribution. And it really drives home

the point that you shouldn't be thinking of your distribution as like a normal shape,

you know, like normal distribution. But it's really, really different. Just forget that

this is a normal distribution, because it won't be. And that's what I really liked. It's not

something you just drop in and then get numbers out and you walk away with the numbers, because

it gives you many numbers. And you really need to understand what you're trying to do.

And especially what you're trying to measure, what your requirements are for your system,

how fast it is actually supposed to be, and how fast is the slowest result supposed to be. And,

you know, it's making you think about the thing you're actually trying to achieve.

And I found that really interesting. And I had a couple of questions to Joachim

ahead of time, because I wasn't sure what this actually means and how to interpret the results.

And he was super helpful, gave me some pointers. And we'll add these to the show notes. There's an

interesting talk about by Jill Tenney, I think is how you pronounce his name,

about why normal distributions, these are normal distributions, why you should be looking at

percentiles and what you should be looking for when you do run these kinds of tests. And yeah,

I think this is really great. This is one of these packages where you start looking at the package

because you think it's useful. And then you end up going down a rabbit hole and learning really

a lot about a domain that you didn't know a lot about. And that really gives me, just looking at

the package and the documentation of it and the breadth of it, you know, like these, all these

different metrics and the care that has been taken to give you controls and knobs and, you know,

allowing you to set these thresholds, make it configurable, gives you really high confidence

that this is a really worthwhile thing to invest time in and use. And I know I've seen this pop up

in a few places, this package. It's been being used in quite a few projects to measure benchmarks.

So I think it's really fascinating. And give this a look if you want to benchmark stuff,

I think this is the perfect place to start. - That's great. Are we going to use it with,

because we have query performance tests in our code base, right?

- Yes, we do. I might try that. There's some, another place where I want to use it first,

but it's certainly something I'd be interested in doing more because right now we're doing really

the bare minimum of just, effectively we're not as much benchmarking, we're sort of snapshotting,

we're baselining, you know, we're trying to establish how fast is this if I run it once,

and then we record that time and we effectively see if we get slower. So we're not really ensuring

that we're hitting a certain speed, we're tracking how we're getting slower over time.

The way I like to look at it is that it's more of an indicator than an actual test. It is,

you should be aware that things have got slower. - Yeah. I think if I was asked what's this for,

I'd say this is to see if we suddenly have a query that runs two times slower, because that we would

see and that would be important. And this has actually happened. And that's what this is for

really to not accidentally, because SQL queries are really easy to just tweak a little bit and

then have huge performance regressions that you don't notice, especially you won't notice it if

your dataset is small. And what we're actually doing is we run it against our staging database,

which has a comparable dataset to production. So we would see if a query is really bad all

of a sudden, we would see that by looking at the, we're not looking at the runtime,

we're looking at the cost, which also isn't perfect because the cost is an estimate,

it can be wrong, but it is a, in my testing, it has been a fairly decent indicator of what's

actually going on. So this turned into a much longer packaged recommendation than usual.

But we haven't finished yet. Cause I was going to say the only, the only slight downside to that is,

is that because that CI step is not a mandatory CI step. What we quite often do is something will

slip by just a few milliseconds and it will start the test failing. And it takes us a few weeks to

maybe get around to updating those benchmark scores. And so if anything doubled the query

performance in those few weeks, we'd never know. Well, actually we would, we'd never know until

we came to update the benchmark scores. And at that point we'd realize, and there we go. But,

but, but yeah, but I mean, who's perfect, right? Yeah. I mean, that's the thing. This is hard to,

to automate because A, we want to, don't want to be running this all the time.

Like auto updating, right? You need to review these new numbers, right? You need to put them

in somewhere. We need to baseline somehow. And because Swift package manager doesn't have the

same affordance to do these, what's it called in Xcode where you can, you can baseline tests,

right? You can have performance tests in an Xcode and baseline them automatically because we can't

use that due to the way we're running tests. We sort of have to do that bit manually. Otherwise

that would be really great if we could do that and then just review the changes and approve them.

That would be ideal. Right now we can't, I mean, the best thing we could probably do is,

is write a, like the Pointfrico guys have done this auto snapshotting thing we could, but you

know, I've already spent way too much time instrumenting this. I think I'm not gonna,

I'm not gonna take it even further. This is, I do this once a week and that's, that's fine. So

it's not too much work at the moment. So my next package recommendation is by

Navin Chauhan. And this is something that I never expected to see in a Swift package. And I was,

I was kind of delighted when I saw it. It's called Swift Gopher. And it is a, a less than

one month old implementation of the Gophers, both server and client written in Swift. Now

I'm pretty confident that there are people listening who will not remember Gopher.

Do you remember it Sven? I do remember it. I saw it early on,

but I actually never used it. I know it's one of these, not ethernet, internet protocols, you know,

like FTP, HTTP, that sort of stuff. But I don't, what did Gopher actually do? It was, I don't

remember. I also, so I did, I think I did use it like once or twice. I remember on the old

Unix machines at university. I think there was still a couple of Gopher. I think Gopher was

mainly over by the time I even got access to the internet. So when I was at university, which is a

very long time ago, the internet was really only just becoming something that you could get access

to even in academical institutions. Certainly there was no home internet access at the time.

And Gopher was kind of dying at that point. So it was really early. But what it was,

was it was a precursor to the web and you could Gopher into a server. And it was like a step

up from Telnet. So Telnet, you basically got a remote console on the server. That's what Telnet

basically did. And Gopher was a way that you could fetch information and then post information back

to it in a text-based interface in the very early days of the web. But it's not dead apparently,

because Navan has written both the client and the server and you can run a Swift Gopher server

and you can connect to it with the Swift Gopher client and you can fetch information from your

Gopher server. It's a bit like these very early BBSs. I have no idea how that actually used to

work. So I do remember those. So that was really one of my first experiences with the internet,

which was the Telnet BBS systems. The only one that I remember by name was a Telnet BBS called

Skynet, which is a very original name for a BBS. And you would connect effectively using,

you'd effectively get a kind of shell into the server and it would be this BBS system that you

could leave messages for people, you could read messages, you could send messages, you could post

on public forums, I think, but they were very, you know, it was a very different to anything you'd

see today, but it was all through a kind of text, a terminal style window. And I remember the speeds

that we were able to have at the time, you could see the terminal screens drawing character by

character. So it was, it was early, early days of the internet. Well, Gopher, I didn't have that in

the cards as a package. I was pretty confident that that would be all mine. Yeah. Well, this

is one of my traditional recommendations, which is not a recommendation. Oh, is that what it is?

Although I do think we should, we should maybe consider supporting Gopher for the package index.

So we don't neglect those people who still want to use a Gopher client to fetch package information.

You can fetch your package of the day. Exactly. My second pick is called WebSocket Actor System,

and it's by Stuart A. Malone. And that's, I loved seeing this. So I've been following

Apple's distributed actors package with interest for quite a while now. But in the past, I've,

I've really struggled. I've tried a few times, but I've struggled with creating a proper example

that would work across nodes. So the repository comes with, with an example, tic-tac-toe the Apple,

Apple's distributed actors package, and I've played around with it, but I've never really

gotten it to work properly in a, in a local setup. And, and Stuart has really done all the hard work

here and created a package and then also shipped an example and explained how to use it. We can

set up a distributed actor system based on WebSockets here to communicate between nodes,

where nodes really is just another distributed actor instantiation. So it's really fascinating

because from a dev point of view, you're effectively just calling into an actor and

you don't even realize or know whether it's local or in another process or on another machine

across the network. And that's the beauty of this, of these distributed actors. It abstracts away

all of this and, and all of this message passing and, and encoding decoding is, is handled by the

library which is really nice. And Stuart has chosen WebSockets here as a transport and I think

JSON under the hood as a, as a exchange format, but you actually see none of that. So you really

just have actors and you, you know, you, you connect and then call functions with try await,

you know, like normal async functions on the actor that you've just resolved across the network

potentially. Now I haven't tested, I've actually used this in a little example to see how it works,

but I haven't tested what this means in terms of efficiency, resilience, error handling,

that sort of stuff. It does claim to automatically reconnect after failure. So it has some

affordance for this. And the other thing that's really nice about this, as opposed to a normal,

traditional HTTP rest API, for instance, is that you can call methods, actor methods on

the service you connect to, but the other service, you know, can also do this. So it's actually

really nice if you want to have a server that calls back to you as a client, you know, imagine

it's like a push mechanism, right? Whereas normally you talk to an API, you'd have to

pull the API to understand if something's there for you, you know, if something has something to

respond. But in this case, these two nodes are really peers. So if there's something happening

on the server, it can actually call into the client and say, look, this just happened, you

know, handle it. So that's really nice. And that might open up use cases that are quite powerful.

So, yeah, I really wanted to highlight it because I'm really interested in the distributed actors,

and I haven't seen a lot of packages actually using it, or even like proper applications using

it. And maybe this package WebSocket actor system can be a building block for someone who's doing

something with it. I have sort of a couple of ideas how we might actually be using it

in the future. That's why I'm super interested in this whole domain. And I've really loved seeing

this package come around. Are you ready to share those ideas? Well, I mean, it's no secret we're

running a build system, right? And right now our build system is... I figured that would be it, yeah.

Yeah, I mean, this because we have components that talk to each other.

And we're currently using GitLab as our queuing system. And if we ever have need to change that,

I'm sort of toying with the idea that rather than building some sort of HTTP-based system,

maybe this would be a good use for actors where the actors are the builders and they report back,

that sort of thing. I have to talk to people who know more about these kinds of systems if

that's actually a good idea, but it strikes me as something that might work really well for us here.

I don't know. I think these things make me overly nervous because they feel... There's no judgment

here on the actual reliability of this because I'm completely uninformed on the actual reliability.

But my perceived reliability is... I'm suspicious. Yeah, I mean, this is why I said I haven't tested

this in terms of efficiency, resilience, and reliability. I think that's really

a key thing to understand. But it's also true that distributed actors is based on tried

technology. So there are actor systems out there already. And Conrad, his last name escapes me

right now, at Apple, who's working on these systems, he has lots of experience with this.

And this is Apple's adaptation of actors. And this is not entirely novel, is what I'm trying to say.

Like, this is not completely uncharted territory. This is Swift's take on a system that exists. I

think Erlang has this in a certain sense. There is certainly a Swift spin to it. But I think we're

not, you know, like complete greenfield here. I think there's lots of reasons to approach this

with interest and see how it actually works and understand, especially if this is a good fit,

right? Because I think it's also important to stress that distributed actors aren't intended

to replace every distributed system, right? There are reasons why HTTP is really good. Because I

think if one reason I can think of where you probably wouldn't want to replace it, if you

need caching, that sort of thing like this or proxying, you know, like HTTP as a protocol

allows you to do lots of things where your services don't even know that they're happening,

because it's just a plain standard protocol. You couldn't, I think, insert anything like that into

a distributed actor system and expect it to still work, right? They really rely on talking to the

other end of the system that is as the same part of the system. There's no expectation of this

being proxied or load balanced or anything like that, is my understanding. Again, I might need to

talk to people who understand this way better than I do, to really get a feel for this.

We just continue with our reputation as the best research podcast in the world.

And with that, I think we should probably wrap it up for this episode. We will be back in a

couple of weeks with some more news and some more package recommendations. But until then,

we will say goodbye for now. Yep. Goodbye. See you in two weeks. All right. Bye-bye.