Instant WebStick - ESP8266 Web Server / NAS

Description

As a full stack developer and a student, many of my friends ask me how to setup their own website for free. Although there are a lot of free web hosting services out there, they are quite picky about service provider and vendor lock in issues. As some of them are also interested in tech stuffs, I will sometime introduce them to self-hosting.

The issue with self hosting a web server is that many of them have no experience in Linux and just putting an old Windows PC with WAMP or Nginx is a waste of space and power. That is why I developed a fun little WebStick for these use case.

What is a WebStick?

A WebStick is a USB stick form factor web server that connects to your home WiFi and allow you to connect it via mDNS in local area network. You can also access it from the internet by changing a few port forward setting in your home router.

Details

Features

Basic Web Server
Directory View
Cookie based login system
File Manager
- File Upload / Download (with multiple-file zip download support)
- File Open, Cut and Copy
- Support UTF-8 filename (32 bit length only, filename length depends on variable bytes used per char)
File Searching Tool
- Much slower than Google Search (Is this even a feature?)
System Information
- SD storage info
- WiFi info
WebApps
- Music Player
- Photo Viewier
- Video Player
- Markdown Editor
- Notepad (Modified version of ACE Editor)

Usage

Hosting

Format your micro SD card as FAT (Recommend using SD Card Formatter)
Download and put everything under sd_card folder in this repo into your micro SD card
Edit the files under /cfg/ to setup WiFi credentials (wifi.txt), management passwords (admin.txt) and optionally mDNS domain name (mdns.txt)
Insert your SD card into the WebStick SD card slot
Power on your WebStick by inserting it into your computer USB port (for debug / flash mode) or phone charger with USB A port.
Access your WebStick using the device IP address (http://192.168.0.100/) or mDNS name (e.g. http://webstick.local)

Using your domain with WebStick

mDNS are local only domain names. If you want your viewer to be able to access your website from the public internet via your domain or your home public ip address, you will need to purchase a domain name, setup your DNS and / or your NAT router's port forward rules.

Build from Source

The following libraries are required for building the WebStick backend code (firmware to be used in ESP8266) using Arduino IDE. You can find the source code for the backend bare metal web-server in the /web-server folder.

Board Settings

Here are the recommended settings for best performance.

Board: Wemos D1 R2 & Mini
- CPU clockspeed: 160Mhz
- IwIP Varient: v2 Higher Bandwidth

Require external library:

- ESPAsyncTCP (https://github.com/me-no-dev/ESPAsyncTCP)
- ESPAsyncWebServer (https://github.com/me-no-dev/ESPAsyncWebServer)
- ArduinoJson (https://github.com/bblanchon/ArduinoJson)

Limitations

As you might have notice by now this system is hosted on a 1 USD MCU (ESP8266), even if web server is written in bare metal, it still have the following limitations

Filename length (hardcoded to 32 bits per filename, might be shorter for UTF-8 as it use variable byte structure, but technically this is not the system limit)
File size limit (recommend) <= 5MB, any file size larger than 5MB that is not stream-able (e.g. mp4) is not meaningful. Stream-able files can be larger than this size (e.g. webm) with acceptable playback speed.
Slow Upload / Download speed (2 - 4Mbps), Very slow concurrent R/W speed (200 - 210 Kbps)
Single admin session (cookie), due to limit in runtime memory (i.e. If you try to login from 2nd browser, the first browser will automatically logged out)

Files

sd_card.zip

HTML for the front-end with API access to the backend

x-zip-compressed - 11.03 MB - 09/02/2023 at 08:38

Download

Gerber_webstick v2.1.zip

Gerber file for the PCB

x-zip-compressed - 131.54 kB - 09/02/2023 at 08:38

Download

esp8266-web-server.zip

Arduino source code for the esp8266 web server (backend)

x-zip-compressed - 14.78 kB - 09/02/2023 at 08:38

Download

Components

6 × 0805 10k Ohm Resistor

2 × 0805 10uF Caps

2 × 0805 100nF Caps

1 × CH340C USB to UART converter

1 × ESP12F (or ESP12E)

Project Logs

Collapse

Adding support for multi-users

tobychui • 03/21/2024 at 13:34 • 0 comments

Adding Blog Engine to WebStick

tobychui • 09/18/2023 at 15:20 • 0 comments

It sound a bit crazy, but I just realized that without changing anything on the backend (ESP8266 code), I could in fact write a blog engine for the WebStick.

Though, it is a really basic one with minimal blog features, but it is working!!!

Build Instructions

Collapse

Supplies

As this is a relatively simple build, you don't need much materials to build yourself a WebStick.

A custom printed PCB
USB A plug
CH340C (USB to UART converter chip)
A few resistors (10k) and caps (100nF & 10uF)
AMS1117-3.3 (Voltage regulator)
Micro SD card socket
LEDs
ESP12F (or ESP12E, both works)
Micro SD card (with FAT file system)
BC817 transistors

Order the PCB

As the PCB is the main component of this build, you can order your PCB at any PCB manufacturer you prefered. This project is sponsored by JLCPCB. If you have never tried to order a PCB before, please feel free to try out JLC services with the link below.

https://jlcpcb.com/

In my case, I got mine for around $5 - 6 USD (5 pcs), which is perfect for small volume productions for making a few prototypes for fun. The PCB should take around 3 days to produce and another few days to ship to your location.

You can find the Gerber files for the PCB over here on my git server.

https://git.imuslab.com/TC/WebStick

Solder the Components Onto the PCB

How to Solder The Components?

This step is also quite straight forward. What you need to do is simply prepare all the components you need for the soldering. Then solder them onto your PCB one by one.

As each of the boards takes me around 30 minutes to solder, after a few successful prototype, I decided to go with JLC's PCBA service. If you are a STEM teacher and needs a bunch of these made like I do, feel free to try out the PCBA service as well. I will have another blog post documenting how to mass produce electronics later if anyone is interested about this.

Why the Components are So Small?

The smallest component, the NPN transistors BC817 are there to support automatic programming. In boards that do not contains build-in programmer like the ESP32CAM, you will often see a FLASH and RESET physical buttons. That is because it is required to held those pins to LOW in order to make the chip to enter flash mode. But adding these two components (although they are difficult to solder), it provide a much smoother automatic programming experience to myself, specially when I am developing the backend code which I need to flash a new firmware to it every few minutes just to test it.

Discussions

Robert Alford wrote 10/02/2023 at 21:51

If I need to Change the Port from 80 to 81 as I already have something on 80 for PortForward what Change will I need to make ?

Are you sure? yes | no

tobychui wrote 10/03/2023 at 01:37

Go to your router settings and set external port 81 port forwarding to internal webstck IP :80

Are you sure? yes | no

Robert Alford wrote 10/03/2023 at 02:44

If it was that Easy I would have already done it.. My Router does not give me the Option of an Internal IP just open the Ports so thats why I was asking but I did find it in the Files that I can Upload directly to the Chip.. I have a better Router on the way so that too will work..

Are you sure? yes | no

allexoK wrote 09/10/2023 at 13:44

Wow, great project! I wonder why didn't you use Esp32?

Are you sure? yes | no

tobychui wrote 09/10/2023 at 13:52

esp32 cost twice as much as an esp8266 when source in bulk. Besides, it is more fun to work with lower end hardware than a more powerful alternative :D

Are you sure? yes | no

allexoK wrote 09/10/2023 at 15:23

Yeah, it makes sense:)

Are you sure? yes | no

Kogije wrote 09/05/2023 at 06:18

Is it possible to modify this to host a Python Bot for Discord?

Are you sure? yes | no

tobychui wrote 09/05/2023 at 07:50

No. This only runs Arduino C++. maybe you can host a web page with python interpreter to do something with python in your browser?

Are you sure? yes | no

Kogije wrote 09/06/2023 at 01:31

I was hoping it could be side-loaded from the micro SD. Oh well.

Are you sure? yes | no

aaaaaa wrote 09/04/2023 at 15:31

meybe add mesh network

like https://freifunk.net/

and add a power ups for example 18650

Are you sure? yes | no

tobychui wrote 09/05/2023 at 02:49

I would love to, but in order to implement something like Apache in the ESP8266 firmware, I already use 98% of its internal memory. Maybe next time I will have a separated project that do mesh networking stuffs :D

Are you sure? yes | no

aaaaaa wrote 09/05/2023 at 12:26

for mesh You not need web server ;-)

simple ssh or less. Node must forward packages/frame and nothing more.

Meybe instal fuzix ?

BTW, power is important. small device is no problem , problem is with power and mobility

Are you sure? yes | no

Instant WebStick - ESP8266 Web Server / NAS

Description

Details

Features