Writing a Filestash plugin

This content originally appeared on DEV Community and was authored by Migsar Navarro

This morning Three days ago I finally got my Filestash plugin working as intended, after a week spending my early mornings working on that. This is just the first step, I am aware of that, and the non-coding part often takes longer than the coding part. The purpose of my plugin is really simple, I wanted to be able to set a back-end account in the configuration, so I don't have to login my s3 credentials every time I want to use the app.

When I discovered Filestash I really loved its simplicity, sadly, as it often happens, I am discovering it is not so simple under the hood, not a bad thing, just to state some often not told truth, product simplicity very often do not correlate to technical simplicity.

The main reason to write this post is to help others in that regard, contribute to make Filestash a bit more simpler technically, or at least easier to understand.

In my previous post, A very basic plugin for Filestash, I described some of the problems I faced when I got started working with Filestash source code, this post builds on top of that.

Why do I need a plugin?

I've been paying for a s3 compatible storage for a few months, and I really need it, but often I skip using it because of the lack of a UI, there are a few UIs already, but none of them adjust to my needs, so when I came to an open source project like Filestash, I thought it was perfect.

Since I don't think my use case is standard, I'll give you a few more details:

• I want to use Filestash as a GUI to manage my s3 compatible object storage.
• I want to upload files, particularly photos and audio from a few different sources, particularly my cellphone (I guess it is not that common to use s3 object storage, and probably it is more common to pay for a backup service or for something like Google Photos but hold on, I'll explain).
• I want to use the files in my s3 storage seamlessly in web-applications, Jupyter notebooks, and all other sort of network based experiments.

In contrast:

• I don't need just a web file manager to move files between computers and the cloud.
• I won't be sharing the access to this storage, I'll be the only one using it.
• I will be sharing, some of the files I have in the storage, without ever showing Filestash UI.

I've have not found a solution for what I need, and if you know about some nice solution please share it, I would love to hear about it. I need a really simple product totally aligned with my needs because any friction will make me not use it.

One last non-technical but very important reason. I'll do the development myself because I enjoy coding when it has a purpose, and I don't have the budget to hire someone else to do it, nor the patience to manage it.

The problem definition

My problem was:

I need to initialize an s3 backend (although it should be similar for other backends) so I can open the app and I don't have to type the credentials for the backend any time.

I want to avoid this screen by automatically login the user

From previous experiences I knew it was a good idea to take a look at the Github's repository issues, to see if other people have experienced or probably solved the same problem. I didn't find something similar to what I need, there was one open issue but it was really new and didn't have any useful information.

The first step was a quick product inspection, I started by opening the dev tools in the browser and taking a look at what happened when I run the app and do all the stuff manually. It was not as linear as I describe it next but it was not hard at all. As an example of the lack of linearity, just two days ago I suddenly realized I had not explored how the demo app authentication happened.

The demo app actually uses a request to /api/session with the credentials from the home page of filestash so the cookie is created. It is a quick solution that could work but I don't really like it because of two things: the first and most important is that it exposes the credentials, something that I didn't wanted to do, the second is that is not elegant at all (of course this is just my opinion).

Getting started

My initial approach was to skip authorization altogether, just initialize the back-end and directly jump to the files view. It may be possible, but I realized that for that approach to work I would need to edit the core, something really want to avoid. So I decided to find a way to make it work as a plugin, even if I already know compilation was needed to configure the plugin.

I didn't have any idea of what the app flow was, I had an idea of how I would architecture a similar application, but that didn't mean there were not other ways to achieve the same things.

From doing things manually I noticed that the first time I was immediately redirected to /login, but there was no login in the routes, so I assumed it was something in the browser. Then I realized that / was taken care at the end of all the routes in a catch all block. When I filled the form and click the submit button a POST request was made to /api/session.

The user needs to send the storage credentials to the server.

It took me some time to discover the logic for the front-end, partly because it is a custom vanilla-js but react-like solution. At some point I read the discussion about keeping a React app or moving to vanilla-js, I don't like the new solution but hey, with open source is like this.

Routing

The router is located in server/routes.go. There are three routes we care about:

• GET /, it uses ServeFrontofficeHandler handler. This is the entry point, I don't want to type a complex path before viewing the content.
• POST /api/session, it uses SessionAuthenticate handler and creates a cookie. This route is used when the user submits the credentials in the /login page, which apparently is just a browser view in a SPA.
• GET /api/session, it uses SessionGet handler. This is the endpoint the home page hits to know if the user is logged in, or more accurately, if the storage has been initialized, and then to decide what view to show accordingly. If the user is logged in the view will be the filesystem, otherwise it will be the login page.

An important thing to notice is that each one of them uses a different set of middlewares. Most of them are not really important, but as I was trying to understand how things worked I found SessionStart and BodyParser were relevant.

SessionStart injects the session for the request context, so even if a backend has been initialized, some session variables won't be available unless the route uses this middleware.

BodyParser injects the body of the request into the context. Not sure why this is necessary, but it took me a while to figure that the absence of ctx.Body was breaking my code.

Hooks: The best place to inject my code

It was not obvious to me where to put my code. The hooks system is not documented, I already knew the possible hooks, but I didn't know what were the entry points for those, also, at the beginning I didn't know how the server persisted the state.

My initial thought was to use the Onload hook, since I wanted to start the backend before any request and keep it initialized, but then I found that this hook does not inject the App to their handlers, so I was not able to save the initialization once done.

Then I thought about using Middleware or HttpEndpoint, to be honest I still don't know if I choose the right thing, even if my code will impact requests I don't really feel like it is a middleware, it should be run just once at the beginning, but there is an advantage of running it as middleware, and it is that I am able to send the cookie to the front-end once I finished initializing the backend.

So I ended up registering it as middleware but running it only once for the given path, this offers some flexibility but I am not totally happy with this solution.

The journey

My plugin doesn't need to do anything new, and I was aware of that, it was more about discovering where things were being done, and then see how much I was able to re-use, the main purpose was to slightly modify the app flow.

My biggest problem was to define what the flow should look like, with the usual flow you have a session because you have some state that is session dependent, in this case, the storage back-end is user defined, but if I want to have a fixed storage back-end I don't see the need for a session, maybe it is important in the server to keep a connection open for some time for the sake of efficiency, but from the user perspective, you don't need a session anymore. The implicit decision in this case is that the logout button that appears on the top right of the UI is not needed anymore.

The login happens in the server without user input

Authentication

This is a very important topic and one that usually is difficult to understand and to implement properly. I am not an expert in authentication but after ten years working with very different code bases I've seen many different ways to achieve the same things. Often in sub-optimal ways, some times I've helped improve the code, some other times I've not been able to convince the team the current solution is not optimal or even faulty, sometimes, I've only been able to detect the flaws retrospectively, after seeing it properly solved in a different company or after working it for myself in one of my projects.

Filestash offers some authorization middleware, and some authentication middleware, but I didn't manage to configure it properly. Even after disabling absolutely everything, there is the need for a session, both for the sake of efficiency and security. In the code it can be read something like isAuthorized but it refers to the authorization from the point of view of the storage service provider, which could or could not be the same authorization that the app will use. As an example, for my use case, I would like to hide the storage provider authorization from the user of Filestash, which will be just me, al least initially.

This storage authorization is sent to the browser, probably for the lack of another method which would be implemented as part of the authorization or authentication middleware, but while I was exploring the code I didn't find flags to disable this to happen or to make it work in a different way but I am not 100 % sure if it cannot be done or simply I missed it.

Internal state

The app saves everything in the App structure which is defined in server/common/app.go and often injected as the ctx parameter. And important note here is that there is a Context field in the struct but it doesn't seem to be used for the session. Here are the four fields that were relevant for the plugin:

• Backend saves a reference to the client used for accessing the storage.
• Body is not used directly, but it is used by SessionAuthenticate by including BodyParser as a temporary storage for the POST body.
• Session stores the credentials for connecting to the storage.
• Authorization stores the same hash that is included in the cookie.

Front-end

At the moment there was nothing to be done, but at some point in the future I would like to add a route and a page to set there the login information and some other session parameters, like deciding in which page or pages inject the cookies, or the home directory for the storage.

The only part that we care about is public/assets/pages/ctrl_homepage.js, this is executed for the home page, that is the / path, and there a GET request to /api/session is done to know if a session already exists.

The plugin does not modify this, but send the cookie in the / request so when the call to /api/session is made the cookie is already there, without ever asking for the credentials, so the files are displayed.

The cookie

Cookies are using to keep track of both the user and admin sessions. There is not much to say in this regard, still I think it is important to create a different section to make it visible. The functions for encrypting and decrypting are included in server/common/crypto.go and the secret used is defined in the settings section of the admin panel.

Conclusion

This is a basic plugin and there are still plenty of room for improvements, the most important thing is that currently this only works for s3 storage, which solves my problem but is very limited compared to the wide range of storage that Filestash support. Another nice thing to have would be a plugin configuration page, but for the moment it solves a problem. The next days I'll work in getting Filestash to work with a reverse proxy (Caddy) and deployed it to production.

This content originally appeared on DEV Community and was authored by Migsar Navarro