Your web server wishlist

[mholt]

I'm the lead developer working on Caddy 2, the next generation of the Caddy Web Server -- and I want to know from you what you want your ideal web server to be like.

Caddy 2 is currently in beta, with a hopeful release candidate at the end of next month. Web servers don't often get huge updates, so this is your chance!

Whether you do front-end or backend or devops or SRE or whatever, I want to hear your wishlist! What will make your life easier?

Feel free to think outside the box, i.e. outside of what your current web server already does -- of course we want to keep what is good -- but what do you wish was possible or easier or different?

 

[tom]

Hi Matt, great to see you here!

A few things that come to my mind (don't know right now if any of these aren't already supported):

• Dynamic error pages with data fetched from a database.
• Backend selection based on location (using maxmind geoip2 database or likewise).
• Caching to redis for static files (html, graphics, javascript, ...).
• Expired header checking for cached files every x seconds/minutes and automatically updating changed files in the cache.
• On the fly backend adding/removing/updating by cli or rest api.
• Endpoint to see if SSL certs are available for a specific domain (if so, return the path to the files so other local services may use them).
• Possibility to force-cache custom routes (so we can use Caddy for uncachable applications or applications that we can not modify)
• ...

 

[Gummibeer]

I have no idea what Caddy ist already capable to do. I'm using nginx. But things I would wish to improve:

Integration of fail2ban. Like possible with nginx and a lot of configuration by using req zones, limits and related log parser for fail2ban. At the end this could be seen as a small DDOS guard (as long as your server can handle the requests in webserver).

Automatic setup and integration of letsencrypt. It's everytime a 3-4 step procedure to add a new domain.

Templates with placeholders. Nearly all of my vHosts have the same configuration. Except the used domains, document root, php-fpm socket and SSL certificate.
So having a template and being able to create a new vHost by simply extending the template and set the variables would be awesome.

 

[tom]

Automatic setup and integration of letsencrypt. It's everytime a 3-4 step procedure to add a new domain.

You absolutely have to try Caddy. This is one of the best things it does. Handling all your certificates automatically for you, without any difficult configs.

 

[Gummibeer]

Have to extend: SSL labs A rate by default. This means only using modern ciphers, protocols by default.
You should be able to downgrade the rating but by default this would be awesome.

 

[mholt]

Great feedback so far! (Thank you!)

I'm taking notes on all of these requests, and I've chosen a few to respond to here.

Expired header checking for cached files every x seconds/minutes and automatically updating changed files in the cache.

Do you mean Expires header? If so, then I think I understand, and this'll definitely be useful. Is one of the many items on our list for the HTTP cache handler: caddyserver/caddy

Endpoint to see if SSL certs are available for a specific domain (if so, return the path to the files so other local services may use them).

Interesting idea -- what is the utility for this exactly? i.e. if a certificate is not available, what is the other local service supposed to do? Do you want to use Caddy as a dynamic certificate manager? If so, would the local services even be able to reload renewed certificates?

Templates with placeholders. Nearly all of my vHosts have the same configuration. Except the used domains, document root, php-fpm socket and SSL certificate.

Can you elaborate on this more? One goal we have is to reduce repetition in the configs where possible. How would you populate the used domains, document root, php-fpm socket, and SSL certificate information? i.e. if we did have templates or template-like functionality here, where would the templates get the data from?

Have to extend: SSL labs A rate by default. This means only using modern ciphers, protocols by default.
You should be able to downgrade the rating but by default this would be awesome.

Oh, good news! Caddy has already done that for a while. Here's my personal site running Caddy 2: SSL Server Test: matt.life (Powered by Qualys SSL Labs) - and here's Mozilla's SSL Config Generator; notice that the "Modern" configuration is the shortest (and frankly, you don't have to configure anything for modern compatibility -- they just add HSTS and turn off TLS 1.2): Mozilla SSL Configuration Generator

I'm happy to discuss all of your feedback and want to make sure we understand and get it right! Keep it coming.

 

[Gummibeer]

Can you elaborate on this more? One goal we have is to reduce repetition in the configs where possible. How would you populate the used domains, document root, php-fpm socket, and SSL certificate information? i.e. if we did have templates or template-like functionality here, where would the templates get the data from?

What I mean is something like twig. Features I would use would be define the template to use (a single level of extending would be enough for me), including snippets (define variables to pass), using variables in conditions and "echo" them in the template.

Somewhere in this forum someone requested a webserver configurable by YAML/TOML this would be fine for me to hold data.

extends: my_template
data:
  domain: example.com
  root: /var/www/com.example

In this case domain and root would get passed to my_template.
There I would like to be able to include a snippet conditionally. For example if no php-fpm socket is defined it will use the default php-fpm configuration.

 

[tom]

Do you mean Expires header? If so, then I think I understand, and this'll definitely be useful. Is one of the many items on our list for the HTTP cache handler: caddyserver/caddy

Yes, that‘s what i meant. 😊👍🏻

Interesting idea -- what is the utility for this exactly? i.e. if a certificate is not available, what is the other local service supposed to do? Do you want to use Caddy as a dynamic certificate manager? If so, would the local services even be able to reload renewed certificates?

If a certificate is not available yet, the other service will show it as „in progress“. In my case it would be a Laravel job that checks if the cert is available and how long it‘s valid. It will be used as part of the setup process status that will be shown to the user. This would also be possible by querying the webserver every few seconds under the specific domain but what if eg. the user hasn‘t pointed the domain to the server yet.

Using Caddy as a certificate manager is a pretty good idea too. So Caddy would habdle the web traffic and create the certs and store them in a key-value store like redis and other services could use the certs too.

 

[Dominic]

As @Gummibeer already pointed out, I would like a config file in yaml or toml.

 

[mholt]

What I mean is something like twig. Features I would use would be define the template to use (a single level of extending would be enough for me), including snippets (define variables to pass), using variables in conditions and "echo" them in the template.

I see, so like config generation. Yeah, we can do this. Do you think you could find a few minutes to open an issue with this proposal, esp. if you can create a full config that you'd like to have? It can be small, but should demonstrate the setup you're looking for.

Caddy 2 has first-class support for "config adapters". In other words, you can "bring your own config" as long as there's an adapter for it. So an adapter in your case might look like deserializing the YAML, executing the template, then outputting Caddy 2's native JSON config.

If a certificate is not available yet, the other service will show it as „in progress“. In my case it would be a Laravel job that checks if the cert is available and how long it‘s valid. It will be used as part of the setup process status that will be shown to the user. This would also be possible by querying the webserver every few seconds under the specific domain but what if eg. the user hasn‘t pointed the domain to the server yet.

Hmm, wouldn't it be better for the two (Caddy + Laravel) to communicate directly, rather than one polling constantly? (Sometimes it can take hours to procure a certificate.)

Using Caddy as a certificate manager is a pretty good idea too. So Caddy would habdle the web traffic and create the certs and store them in a key-value store like redis and other services could use the certs too.

It can already do this, but the issue is getting the other services/apps to properly use and serve those certificates, including reloading them when they've changed. Generally I recommend -- strongly -- that the apps have embedded certificate management (like, using a library) for best reliability.

As @Gummibeer already pointed out, I would like a config file in yaml or toml.

Cool -- tell me more about this. Why do you like YAML/TOML? Is it to integrate with existing tooling? Or hand-written appeal? Are you automating the management or generation of these configs or doing it by hand?

Caddy 2's native config format is JSON, which I believe is equivalent to TOML and a subset of YAML, so the translation between the three can be done really easily thanks to Caddy's config adapters. But the native JSON exposes a lot of control surface for power users who need it. Are you looking for a watered-down version of this that you can alter by hand?