top of page
Search
  • Writer's pictureFrederick Dopfel

Experiments with Docker - Building my own Search Engine

Whenever I am talking to fellow “makers” and tinkerers, I find that they invariably fall into one of two camps. Those that love Docker, and those that hate Docker. Generally, I’ve found tinkerers who come from the software side tend to be more pro-docker, enjoying the ease of deployment and the ability to share hardware resources across multiple applications. On the other hand, tinkers who are self-taught (such as myself) or come from the hardware side tend to lean more anti-docker, preferring to run each application on its own dedicated hardware. 


I prefer to run each appliance on its own dedicated hardware largely because it reduces the complexity of software troubleshooting. If some new service or appliance I am running breaks something, I can simply cut power to the device. However, as Raspberry Pi development boards continue to face supply shortages and my server rack is getting full, I thought now would be a good time to start experimenting.


In order to host my media libraries, I heavily upgraded my Synology NAS to have a quad-core Ryzen processor, 64GB of RAM, and 1 TB of SSD cache. Because this NAS is online and connected to the internet 24/7, it makes a perfect candidate to host my docker containers. 


My first docker project is one that I have wanted to build for a long time: a search engine! There are two ways of doing this - either I could build a “crawler” that indexes the internet and searches through that index for results, and an “aggregator” which anonymously polls other search engines for results, then aggregates them into a single results page (without ads or other trackers of course). 


I chose to use the open-source SearXNG search aggregator for my first project since it should be an easier implementation and not require the same level of storage and compute resources as a full crawler and indexer. Synology offers a Docker container management tool, so I simply downloaded the SearXNG container and told it to run. Easy as pie, right? Wrong! Yes, the container runs, and pointing to the correct ports is easy enough, but I don’t have a way to customize SearXNG the way I want. 


In fact, I was shocked to learn that you can’t even access the file system of a docker container while it runs! Instead, you have to have an identical file and “mount” it in the same location inside the docker container to make any edits. For example, I wanted to replace the generic SearXNG logo with my own (admittedly terribly named) “Dopfoogle” logo. This normally would be a trivial manner, either editing or pasting over the existing .png file. However, in this case, I couldn’t just browse a file system. Instead, I had to search online to find where the file was stored and then mount my own file in the same path. 



So yes, Docker does make it easy to deploy apps and containers. After all, it was designed specifically to deploy code developed on one device across a variety of others. However, it isn’t built for tinkerers. Docker assumes you already have expertise in maintaining and deploying large-scale software projects. It is not well designed for people who just want to tinker and mess around with things. Building Dopfoogle was a fun experiment, and I actively use it as my default search engine (you can set custom search engines as the default in Chrome and Firefox). I like that I can see search results without ads, easily access cached versions of pages, and tailor my results more towards scientific papers. However, this experience with docker has reinforced my preference for hosting on dedicated hardware. I feel like I learn more, can customize the tools and troubleshoot better, and have more fun in the process.


As one of my next projects, I plan to self-host servers for some of my favorite games for me and my friends to play together (without paying a third-party hosting fee). Although I could use docker for these projects, I think running these game servers on their own dedicated hardware would be more fun than alongside other containers. 

12 views0 comments

Recent Posts

See All

Comentarios


bottom of page