streams Documentation

An introduction to federated social networks and the Fediverse, with a shameless plug for my favorite system.

What is a federated social network (and why should I care?)


I'll assume you have used some form of social media or online social networking system (Fecesbook, X, internet forums, etc). Most of these are centralized monolithic systems: one entity, be it a giant corporation or something smaller, keeps (and generally owns, sells, etc.) all of the data (posts, user information, images and so forth) on one or more computers that are under its  control. You can't post a Fecesbook message from X, and you can't comment on a YouTube video from Instagram.

In contrast, consider e-mail. There are many e-mail providers, but there is (usually) no problem sending mail from your employer's corporate e-mail server to other people who may use GMail or a university address. All e-mail systems have agreed on a communications protocol for sharing data. This is commonly considered a decentralized system.

So let's combine these ideas. YouTube and X agree on a communications protocol so they can exchange posts and videos and comments.

Ha ha ha, no. They haven't, and there's no indication they ever will. But smaller groups and individual software developers have been doing exactly this for years. The concept is called federation. Various independent systems, which might resemble Fecesbook or X or Instagram or nothing ever seen in commercial social media, have federated and agreed to exchange information via standard communications protocols.

What's more, there isn't just one X-like system, there are literally thousands, of various sizes, operated independently, with their own rules and their own local culture. Ditto for systems that are Fecesbook-like, Instagram-like, etc.

This whole mess of federated social networking systems is frequently called the Fediverse.

A single independently-operated system is called an instance, and may have any number of users.
If you really want to know why "instance", click here.
"Instance" is a computer science term that means, very roughly, "one copy of the software, running on one computer system". Because programmers are as easy to herd as cats, a Fediverse instance is also sometimes called a node, site, server, hub, pod, host, etc.

There is only one instance of Fecesbook, even though technically it's running on a whole lot of computers. But they all function as a unit, as seen by someone using the Fecesbook app or website.

In contrast, there are over 10,000 instances of the popular Fediverse software Mastodon. Each is operated independently, but with the capability to share posts and comments and photos, not just with the other Mastodon instances, but also with thousands of other instances running dozens of other sorts of Fediverse software.


Okay, that sounds nifty, but... so what?


If you think about this for a minute, you'll realize that the Fediverse cannot possibly run smoothly. Thousands of instances means thousands of individuals who are the bosses of one little piece, trying to cooperate with all the others. In addition, the different types of instance — that is, the different software they've chosen — have different visions of how they want to operate. A video-sharing instance doesn't want to look like a blogging instance, and vice versa.

And there are more subtle issues. For example: If somebody posts, can they control who is allowed to comment? This sounds like a small thing, but it leads to big differences in the community atmosphere. What tools do you have to control unwanted comments? If you, personally, can easily decide which comments appear with your post, you may have fewer participants in your conversation, but less trash to take out. But if it's difficult or impossible for you to control comments yourself, you're more likely to ask the instance administrator to block troublesome people, or to break federation with any instance that hosts such people — which affects everyone using either instance.

And there are quite a lot of instances that have "defederated" from other instances. You generally can't get to the entire Fediverse from just one place. At the very least, administrators will ruthlessly block sources of child sexual abuse material. Many go further, and defederate from instances that host members with political or social values they consider to be "bad".

Also there may be financial, legal, and other reasons for larger publicly accessible instances to defederate (good, bad or indifferent). A couple notable examples of such 'defederated' (or "walled-garden") instances would be Gab and Truth Social. These platforms are Fediverse software at the core, essentially they are customized versions of Mastodon. However, they have no connection to any of the thousands of other Mastodon instances or the dozens of other compatible Fediverse software platforms. They are run, more or less, just like any other monolithic bigtech social media platform.

When will you stop telling me how annoying these federated systems are?


OK, OK. Here's a couple things that make this federated network idea so great.

####Freedom of Speech
Real, actual freedom of speech. Or as close to that as possible for an online platform.
To use an oft quoted phrase: with freedom comes responsibility.
If you exercise your freedom to be an ass, thousands of individual administrators and millions of users will exercise their freedom to block you.

But nobody is forcing them to do it.
There is no central authority who decides what may be said and what may not be said in the Fediverse.

On practically any bigtech platform, it doesn't take much to have a post removed (or otherwise censored), get your account banned or even lose it and possibly any of your own data that was part of it. One can't write something sarcastic and humorous about fake news, because their AI will censor you for spreading fake news. Post about any product, specifically or generally, and all you'll see for days are every related company's targeted ads (that's part of how their service is "free", BTW). Don't write about medical/health issues, even in non-public posts. Criticize your government... fuhgeddaboudit... Par for the course on any bigtech platform is a corporate algorithm feeding you "suggestions", hiding friends' posts from you, and hiding your posts from your friends — whatever maximizes user engagement and advertising revenue.

No such issues exist in non-commercial federated social networks. (I mean, the Fediverse has plenty of its own issues — but not like those.)  

Politics and religion are always a can of worms though. I think part of that problem is the social conditioning that says "the two things to never talk about are politics and religion"; instead of how to have a civil discourse on topics which people tend to have strong beliefs, particularly when often those are based more on emotion than logic. Usually a couple little things like "reaching mutual understanding" and "agree to disagree" tend to go out the window.... But I digress.

Bottom line is that the only free speech platform is the one you control. While some large commercial platforms may claim to support free speech, it's more of a marketing slogan than anything. Even X still has issues despite Elon Musk's commitment to "freeing the bird".

Freedom of Association


You're free to choose one (or several) instances that offer the features and rules that are important to you. Yet still interact with people from thousands of other instances that may have different features and rules. I think it's more like the natural/organic social interaction that occurs "offline" than Fakebook (see what I did there? ;) ) and the other bigtechs.  

Control & Privacy


The other big thing I like — well, I'll call it a "DIY algorithm". I can tune my inbox to show me the material I care about, and filter out what I don't want. This, of course, is highly dependent on which software my instance is using, and what tools and settings are offered by this software. - @billstatler@forum.statler.ws


The algorithms used by Fecesbook, X, Truth, Gettr, etc. for the feed/wall/timeline/whateveryouwannacallit I find very annoying. Besides the ads, spam/scam posts, and other junk they dump in there; I might see some of the same posts over and over for a week or more which I've had no interaction with at all, while posts I have commented on and since had other comments don't show up anymore; or something I scroll past and decide to go back to just disappear completely, never to be found again; etc., etc.


One of the biggest things I think is missing (or greatly lacking) in the bigtech platforms is privacy controls. I suppose some have gotten a bit better with this but overall still lack granularity and just seem more klunky to use because such features were added on, not designed into the system to begin with. For more on this see the long list of features below.

You're about to tell me about your favorite Fediverse software, right?


Right. We're getting there...

For a few years I had a saying - "What is this book of faces you speak of?" At some point I finally broke down and created an account there and was active for a few years. Now I very rarely, if ever bother logging on to what I now call Fecesbook. In the early 20-teens, I discovered a little known software known as the diaspora project, which IMO was a better software for this whole social media concept (even though I never seriously got into using it). Diaspora is now one of the many Fediverse platforms/softwares available. I haven't looked at it since I spent many frustrating nights setting it up on a server back then, so I'm not sure if I'd recommend it now. Over the past several years I've setup (and taken down or lost in one case where my VPN provider just disappeared from the internet without warning) several of my own Hubzilla (and more recently streams) servers because I believe this is the software of the future.

Hubzilla is more of a "do-anything" system with social media built in. Wanna have webpages, calDAV calendar, cardDAV contacts/addressbook, webDAV file store (public or private sharable), blog/articles, a wiki, a shopping cart, social media connections and more all federated with the ability to have any users from a compatible system automagically login to your site?
Hell yeah!
Why isn't every independent creator, freelancer, small business, etc. using Hubzilla?
Various reasons I suppose. Too many options and the fact that it can be fully customized and used in place of half a dozen webapps, maybe? People tend to go with single purpose solutions for all these various things and/or take the advice of some web designer that knows the handful of popular commodity products (WordPress? which actually has an ActivityPub plugin now) they are familiar with. Anyway, Hubzilla can be overwhelming and I guess isn't the most "user-friendly" system to just jump on and whip up a full-featured site with because it is more like a framework. Migrating my own website (basic PHP and archived blog stuff from when it ran on Odoo) into Hubzilla has been on my todo list for years now.

Though my ongoing debate with myself of whether to use it or the more simplified, social media centric version known as the Streams repository continues.

Technically, this software has no name. The lead developer has chosen to release it into the public domain with no branding, so that anybody can use it however they want. (My own Streams-based instance identifies itself as , not Streams.)

The Streams repository is the latest branch of a software tree (which Hubzilla is part of) that dates back to 2010, so it incorporates a lot of experience about what works well for social networking, and what does not. It relies heavily on permissions — i.e., what you allow other people to do to you. Can they send you a direct (personal) message? Can they see who your connections are? Can they reply to your posts? Can they even see all your posts?

Most other Fediverse software can't do this, and relies on blocklists to prevent people from bothering you — after they have already bothered you. A Streams-based instance can do blocklists too, but the permissions system is more of an allowlist.

Let's skip the long list of features right now. I want you to understand the implications of a permissions-based system. Quoting the lead developer, Mike Macgirvin:

I often use the "girl in a bar" analogy when describing this. You don't walk up to a girl in a bar and tell her you plan to follow her home and watch everything she does. You give her your phone number or email. (e.g. "I am willing to share stuff with you and I'm giving you consent to communicate with and share with me.").

She gets the right of response and can reciprocate by calling you or emailing you back — once again signalling "I am willing to share stuff with you and I'm giving you consent to communicate with and share with me" from her point of view. And the game is on.

This is how relationships work in the real world. Everything is consent based. In the real world stalkers get restraining orders and go to jail. That is not the best or most suitable model for social interaction on the web. In Streams we do not call this a 'Follow request'. It is a 'Connection request'. ...

In Streams software, we encourage you to detach from the "Followers/Following" mindset and focus on permissions or what your boundaries are and how you allow this person to interact with you. This may be quite different between different people based on who it is. It is not a black and white decision like the followers/following model. There are a thousand shades of grey.

Just like real life.


You want the long list of features?
Conversations:
Communicate directly with the people in the conversation, not have completely isolated conversations with your followers and their followers shouting at each other — and neither audience seeing the responses of the others.

Permissions:
If you haven't been given permission to speak, you aren't part of the conversation.  If you have not been granted permission to view a photo or video, you won't see it.

Audience:
Your choices go far beyond public and not public. Yes, we have groups. We also have circles. You can also just select a dozen people right now and have a conversation only with them.

Algorithms:
You can install them if you want. You can remove them. You control them and can tweak them.

Guest Pass:
Provide special guest access to private resources and media — on your terms.

Nomadic identity, amalgamated identities and single sign-on:
Instance and project/product boundaries don't exist. It's one big space and you are you — no matter what service or services you use.

Post limits, photo limits, poll limits:
None.

Rich content:
Use markdown, bbcode, or HTML. Any of them or all of them.

Cloud storage:
Built-in network file storage integrated with federated access control and social networking access/permissions. Available over WebDAV.

Events:
Calendar and attendance; automatic timezone adjustment.

Delivery Reports:
In a decentralised multi-platform world, stuff happens. Instances and networks sometimes go down. Project developers sometimes introduce bugs and incompatibilities. This allows you to determine what happened to your post or comment and where it actually went once you published it.

Failsafe:
Because the best time to have a current backup of your data is 10 seconds ago. Clone your online identity and content to multiple instances using the Nomad protocol and mirror any changes in near realtime. Then, if your chosen instance goes down (either temporarily or permanently) or you get booted off of it for some reason, your online life doesn't have to come to an end or force you to start over. All your friends and all your content are available on any of your cloned instances — at any time.

Extend:
Change or upgrade your software functionality as desired by installing additional features from addons and the free app collection.

Rules:
You make them.


More Streams-related resources


Getting an account on a Streams-based instance
  • If you want an account without operating your own instance, see this list of Streams-based instances that allow registration. Or since you're here I may be able to give you an account on my server.
  • If you want to be the administrator of your own instance, but want somebody else to handle installation and updates, feel free to contact me for a managed hosting solution designed just for you, or check out the options at managed hosting from K&T Host.
  • Want to host a Streams-based instance yourself and have complete control? Installation instructions are here. If you prefer, you can use Docker or YunoHost. Or my Docker setup can be found here. The Dockerfile as of this writing builds an php8.3-fpm-alpine image (approx. 225M). The compose file is for postgres database, nginx/fastcgi behind traefik which handles the SSL and includes offen/docker-volume-backup for automated offsite backup of db&app volumes


Does this planet's atmosphere support human life?


They didn't call it the Fediverse because it's all the same everywhere. I think of it as thousands of planets (the independent instances). Many are in alliances based on common technology, culture, or politics. The atmosphere can be similar or very different from one planet to the next, and some may seem utterly alien compared to each other.

So if someone asks me "What does the Fediverse feel like? Is it polite, is it hate-filled, is it right-wing or left-wing? Is it full of creativity or full of assclowns? Is it better for short posts, long essays, photos, music, or videos?" — all I can say is, yes, it's all of the above, and more.

Like any group, most of the people using federated social networking are basically good folks. The good folks don't always have the loudest voices, though. So your experience will be a lot different depending on who you choose to interact with, and how you choose to interact.

Those are your personal decisions. But your ability to implement your choices depends a lot on the tools available to you, as provided by the software that your instance is using. Here is a gigantic list of instance software (and even this list is incomplete, because it omits systems that don't use the ActivityPub protocol to federate).

But, how to choose? Some people say: Just pick the most popular software, which is Mastodon. In fact, just pick the largest instance of Mastodon. Here are today's most popular posts, here are the people with the most followers, we'll protect you from "bad" stuff — and we'll explain federation to you later, when you're old enough to understand.

Well, I guess some people prefer that, but to me it's not useful. I don't need the most popular anything. I need the software, and the instance, that best provides the experience that I want. A Streams-based instance works for me, and it's irrelevant that it's way down towards the bottom of the popularity list. If your needs are different from mine, check out some of the other options. But I think that a Streams-based instance is a good "home planet" for starting your explorations.

To explore strange new worlds...


So you've got a new account on some federated instance, and you don't know anybody. How do you find people and connect to them? There isn't a universal directory or search function for the entire Fediverse — that would require massive resources that don't exist in a non-commercial space (and many people would refuse to be listed anyway).

For starters, you probably know somebody. The one who's been nudging you to get an account for the last year? If you can tolerate them, connect to them first, and see who they're talking to.

Your instance may have a directory of users. It will only cover users known by your instance, but that can be thousands of people. Search their profiles for topics and keywords that interest you. (Some instance software will do this automatically, based on topics and keywords you've put into your own profile.)

Some instances offer a feed of public posts. For example, mastodon.social (a very large Mastodon instance) offers "posts from across the social web that are gaining traction today" (an inaccurate description, because it's actually 99% from liberal-leaning Mastodon instances, not the whole Fediverse). Check a variety of instances, and you'll probably find a few people you'd like to connect with (and many others who are so disgusting that you'd like to launch them directly into the nearest star!).

And there are public directories such as Fedi.Directory and fediverse.info that can be searched by topic for some suggested connections.

Talking with aliens — How do I do that?


You can get more out of the Fediverse if you understand the concept of federation. Here's an example.

Suppose you have an account on one Fediverse instance, but you're browsing public posts on a different instance. Maybe you're looking at a beautiful photo gallery on pixelfed.de, and you want to say "Wow!", but without making a permanent connection to the photographer. Or maybe you just connected to someone today, you want to comment on a post they made last month, but that post is only visible on their instance and not yours. How can you comment, when you don't have an account on the alien instance?

There are currently two options:
  1. Your home instance needs to fetch a copy of the message from the alien instance, so that you can reply from home, OR
  2. The alien instance needs to recognize who you are, and give you permission to write a reply directly on their site.


Got that? (It took me 4 years of Fediverse use before I did!)

Your home instance and the alien instance are totally independent entities. Neither one contains all the posts in the Fediverse, and neither one knows all the users in the Fediverse. But federation allows them to exchange the information they need, so that your reply will be accepted and will go to the right place.

How you make that happen
So, option 1 — fetching the alien post — is a little bit nerdy:
  • Go to the alien instance, and copy the URL of the post you want.
  • On your home instance, there will be somewhere to paste this URL. (For my Streams-based instance, it's simply the Search box at the top of every page.)
  • Your instance should find the alien post, and display it for you. Then you can write your comment.
  • You should only need to do this once. Your instance now knows that you're involved with that post, and any replies to your comment ought to be delivered to you automatically.


Option 2 — getting permission to reply on the alien instance — is easier, but isn't universally supported, and the details vary. For example, an alien Pixelfed instance will let you "Sign-in with Mastodon" (if your home instance runs Mastodon). Once you're signed in, you should be able to comment just as if you had an account on that Pixelfed instance. (And as with Option 1, you should start receiving any future replies via your home instance.)

It's even easier if both instances are based on the Streams repository or its close relative Hubzilla. You don't have to do anything at all! If you are logged in on your home instance, when you visit any compatible alien instance, you will be automatically logged in there. (We're hoping that this "Magic Sign-On" feature will become widely adopted, but right now it's still rare.)


But why should I go to all this trouble?


We're used to commercial social media, where almost everything is done for us — but without giving us everything we want. It's free, but our experience is finely tuned to maximize someone else's profit.

Non-commercial federated social networking is different. It's run by thousands of volunteers, it's funded by donations of time and money, and your role can be much more active if you choose. You are handed the tools for controlling your own experience. You aren't just a user, you're an active part of the network. This requires some effort, but the result can be much more satisfying than what any commercial social media can provide.




Last revised: 2024-09-01
Adaptation Author:@tommyj@
Original work Author: [url=https://forum.statler.ws/channel/billstatler]@billstatler@forum.statler.ws

License: Public Domain