Introduction

Welcome, everybody! My name is Alan. I am the owner of YGOPRODeck. I don't usually write articles here on YGOPRODeck. Still, I figured this might be an exciting topic to cover, particularly for any devs going through the pain of overhauling old websites! This won't be an in-depth technical breakdown of the overhaul but more of a general history of the technologies and implementations used on YGOPRODeck since its inception.

I started YGOPRODeck during the summer month of July 2016, I had recently finished college and wanted to keep working on my pretty much non-existent coding skills. After reigniting my interest in Yu-Gi-Oh! a couple of months prior thanks to a friend, I wanted to try to build something revolving around Yu-Gi-Oh. There seemed to be a massive black hole for looking up Yu-Gi-Oh! decks online which honestly still shocks me when I think about it. 2016 was deep into the lifespan of Yu-Gi-Oh! and yet beyond Reddit, there was very little in terms of online resources. Thus, I decided to create a website that hosted Yu-Gi-Oh! decks! Easy, right? If only I knew... 

A quick note before we begin. YGOPRODeck is more than just me. It's an accumulation of myself, my excellent team of article writers, Felix who maintains the deck builder, the amazing users and so much more. This article however will focus on my efforts in building and maintaining the website since it's inception.

Humble Beginnings

A lot of people ask me how did I come up with the name "YGOPRODeck" and it was basically something along the lines of "YGO (Yu-Gi-Oh!) PRO (Professional) Deck (Decks)". I had around ~10 other names that were in contention but I chose YGOPRODeck in the end. 

I decided to start off by using WordPress. This seemed to be a popular-ish CMS (Content Management System) that was highly configurable to suit whatever needs you may have. Looking back on things, this is the biggest factor I wish I had thought harder on as this sole decision made maintaining the website exceptionally difficult as time went on. I spun up a virtual server with a local hosting provider and got to work. I had the bones of a basic site by the end of the week using WordPress with a site theme called Colormag. 

Once I had the basics done I brought it over to the Reddit Yu-Gi-Oh! community to see how they would feel about it. The original Reddit thread seems to, unfortunately, be deleted but the follow-up is still live. The Reddit community was fantastic! Honestly, if they responded differently I don't know would a YGOPRODeck exist today so I've always meant it when I say how grateful I am to that community. 

I want to be frank, the site sucked. It was absolutely terrible in the beginning. Decks were manually uploaded! I can't even begin to comprehend why I thought this would be a good idea or be any bit sustainable. A user would send in their deck list to me via a text-area form on the website. I would then take this list and manually create a page for that deck which involved inserting each card image and creating the list of each card in the deck. It was exceptionally labor-intensive to upload even a single deck. However, back in those times, I would be lucky to receive 5 days in a given day. 

I remember during these early days checking Google Analytics to see roughly ~8 users on the website at one time and getting excited at the prospect of it breaking 10 concurrent users! Somehow, as time went on, the site continued to grow and grow. I eventually knew that the site would need some sort of automatic deck uploader and while my coding skills were still exceptionally raw and unnurtured, I decided to try my hand at it. 

Time to Automate

This was where I needed to start learning. Not only did I need to start learning, but I also needed to start learning fast as the number of decks being submitted was now increasing at an unsustainable level. I looked into a few "out of the box" WordPress solutions but in the end, decided on a custom solution. After about a month of coding (and a lot of conveniently ignored errors after the fact), I had an option on the website to upload a deck and have it appear right in front of you. This was probably the first biggest change to come to YGOPRODeck. I could now actually sleep and wake up to see a whole host of new decks submitted to the site. It was such a refreshing change for me and took a big weight from my shoulders at the time. 

A Host of Changes

From here on out, a lot of things changed with YGOPRODeck. The two biggest changes after the automated deck uploader were massively speeding up YGOPRODeck by implementing PHPFastCache on a deep level into the site and the creation of db.ygoprodeck.com. Let's discuss the latter.

The subdomain db.ygoprodeck.com came as a result of needing (wanting?) a fully functional Yu-Gi-Oh! card lookup database on the site. I didn't want to go near WordPress and all the modifications needed for it so I decided to go the custom route again and try my hand at creating a completely custom card lookup tool. I wanted this site to allow quick and easy searching of cards along with integrating other cool features like pricing, similar cards, compatible decks, and more. It was definitely a labor of love and while I may not have 100% reached the point I wanted with it, I'm still very proud of what I have done with it. 

After a period of a few months of continual coding and testing, I finally managed to get it out the door. You can read about the release of it here. 

The Forums

I have always liked the idea of forums. I'm not sure why exactly but maybe it's because I grew up with some of the early gaming websites and have fond memories of frequenting their forums. As such, a time came when I figured adding Forums to YGOPRODeck would be a really good idea. Of course not knowing how difficult and cumbersome this would become to maintain. You might be noticing a pattern here...

After an excessive amount of research, I decided to go with PHPBB. This software has a long history with plenty of plugins and documentation online that makes initial setup quite easy. Setting up the actual forums was the easy part but integrating it into YGOPRODeck itself was a completely different story. PHPBB obviously uses its own login/authentication/user system and I absolutely did not want users to have to use two different accounts. I needed a method to sync the accounts and maintain the session state between both the forums and the main YGOPRODeck website. Then I found it, a magical piece of middleware software called BridgeDD developed by Dion Designs. This software essentially keeps a constant synchronization between WordPress and PHPBB. The idea is to disable user registrations on WordPress and have users register on PHPBB. The middleware then syncs the user across into WordPress and the plugin will maintain a login session between both the PHPBB forums and the given WordPress website. 

It was a sight to behold. It simply...worked. I also want to give a big shout-out to Dion from Dion Designs. This guy is a genius! Not only did he develop BridgeDD but he also added a lot of custom functions exclusively for me that I needed to be added. Whenever I had an issue or needed some weird specific use case Dion was always there to accommodate. An excellent individual. 

So now we had a working Yu-Gi-Oh! website, a card lookup subdomain, and Forums that are integrated into the main site. I was right where I wanted to be, at least, or so I thought. 

Maintenance Nightmare

Everything seemed to be cozy. I had everything up and running and users were happily using the site. However, there is always something looming on the horizon and what was awaiting me was the already passed EOL (End of Life) for the version of PHP I was using. When the site first launched I was using PHP 5.5 and by the time all of the above was completed PHP 7.0 (PHP didn't have a 6.X versioning) was reaching EOL! I needed to upgrade, ASAP! Not only would it come with a host of security benefits but it would also increase performance as PHP 7 seemed to be outperforming PHP5.X by quite a large margin. I figured it would be as easy as simply swapping the site over to PHP 7.1/PHP 7.2 right? Wrong!

When upgrading PHP, you normally need to checklist a few things to make sure nothing break. Normally most upgrades go fine but with the setup I was running it had turned into a nightmare. Frankenstein's monster you might say.

WordPress has a few moving parts to it. It has the WordPress core itself, it has the plugin you might be running, and then it has the theme you are using. Right off the bat that is 3 different things that can break immediately when you upgrade PHP. The Core and the Theme are easy enough since they are often maintained (unless you are using a dead/custom theme). It's as simple as updating both of these when you migrate to a new PHP version. The plugins however are a different story. At one point in time, I believe I was using 40+ different WordPress to accomplish different feats across the site. Something as simple as tracking views across different decks for example would be a common use case for a WordPress plugin. I am going to hold my hands up and say it's honestly a lazy approach. Felix (who runs the deck builder) once said to me that I absolutely don't need to be running so many plugins and I knew it. I just didn't want to sit down and attempt to manually code the integrations into the site. I hold my hands up!

However, this now meant that I was using some plugins that were essentially dead. Abandoned by the original plugin author and as such not compatible with future versions of PHP. These plugins would now completely take down the site if left active while I updated to the latest version of PHP so something needed to be done about them. I remember sitting down for about 2 weeks and going through each and every plugin to see which ones I explicitly needed and which ones could easily enough integrate manually into the site. By the end of the exercise, I managed to cull a large number of plugins and was finally able to upgrade PHP to a new version! But once again things were not as straightforward as they seemed.

Remember how great it was to integrate PHPBB into WordPress? Well, that also needs to be maintained. You might be thinking that it should be a simple case of upgrading PHPBB to a new version? Unfortunately, it was not. the magical piece of software I was using called BridgeDD also had target PHP versions that it was compatible with. So if I updated PHPBB to a new version to be compatible with PHP7.X then there was a high chance that BridgeDD would break as the version I was on was only compatible with 5.X. Dion however was continually working on BridgeDD and a version did release that was compatible with PHP7.X. It was then a test of my patience to carefully disable specific segments of the website piece by piece, upgrade each piece individually and then bring them back online all while waiting in hope that they wouldn't break anything. It was a horrible experience to blunt but we were finally on a higher version of PHP.

Change Required

This should bring us up to roughly 2020/2021. The site was exploding in popularity thanks to some new tools I created such as the Pack Simulator (Reddit thread here) but the maintenance routine for the website was completely unsustainable due to the three-piece web I had somehow gotten myself tangled in. Not only that but the site was starting to look old (was it ever modern looking?) and its performance was nowhere near where I wanted it to be. Adding new features directly into the main YGOPRODeck site was cumbersome due to WordPress and it was becoming unfun to work on. Something needed to change. 

Unexpected Routes

I knew something needed to change regarding YGOPRODeck and its setup and if not I could see the site dying a death which would have been really unfortunate. I needed to somehow make a site that did everything above in a better more efficient manner. I wanted to create it myself from scratch so I could easily maintain it and keep it as straightforward as I want it to be. There was no time like the present so I got to work but funnily enough, it was on YGOPRODeck.

Digimon had recently released a brand new TCG and I figured creating a site for it would be the perfect chance to hone my skills and get myself ready for a YGOPRODeck overhaul! I wanted to plan a project out from the beginning, check what I would need from a site and see if I was capable of implementing them. I made myself a small checklist that went something like this:

1. Easily Submit/Edit Decks

2. Easily view Cards

3. Integrate useful deck tools (pricing, hand tests, etc)

4. Implement an in-site forum system

5. A Pack Simulator

6. A Collection Manager

7. Create/Edit Article Content

If I could get the above done in a skeletal site, then I would have the basis of some sort of custom CMS that I could tailor to my specific needs. After working on it for a couple of more months, digimoncard.io was born! 

DigimonCard was the first site that really made me believe that an overhaul to YGOPRODeck was possible. It finally had everything I wanted in a manageable and maintainable system that I fully understood. Naturally, it wasn't all plain sailing and there's been quite a few bugs and enhancements to it from then right up to this very moment where I have fixed some more bugs to the CMS system this morning. I suppose you would always think that when I completed DigimonCard that I would finally move on to the overhaul of YGOPRODeck? Not quite. The following came to fruition:

1. pokemoncard.io

2. marvelsnap.io

3. shadowcard.io

4. rushcard.io

And more! I just found myself continually iterating on this CMS system that I created and it allowed me to push out sites for other card games that I had an interest in. Looking back, the work on all of these sites help me massively when it came to the YGOPRODeck overhaul as I was continuing to gain knowledge with each iteration.

Decision Time

We now bring ourselves up to October 2021. I finally decided to sit down and make some attempts at an overhaul for YGOPRODeck. It's time to start nailing down what needs to be migrated but more importantly how I will migrate. I needed to combine my web of components (WordPress, PHPBB, db.ygoprodeck.com) into one unified site. As you know by now, YGOPRODeck was made up of many different moving parts that made it difficult to pull information out in a cohesive manner. I needed to be able to acquire the users, decks, articles, comments, and more from the site without breaking said content in the process. The task was looking to be enormous and I would be lying if I said I wasn't being put off by the amount of work required. 

For anyone who has looked at a WordPress database before will know that it's made up of many different tables for taxonomies and categories and so on. I had to come up with some clever MySQL queries to all at once pull the content I needed in a timely fashion. At the time of conversion, we had over 50,000 decks uploaded to YGOPRODeck! I needed to be able to process each deck individually without causing strain on the server. Of course, the magic word when looking for database performance is course, indexes. For those who don't know a database index is a data structure that massively improves data retrieval query speed at the cost of storage. A few indexes later and bingo, we had some good queries to pull our data.

After a couple of weeks of tweaking, testing, and more. I had a script that fully converted decks on YGOPRODeck to the format needed for my new custom site. This granted me a good opportunity to actually trim out the many (easily over 1,000) invalid decks that had been uploaded to YGOPRODeck since 2016. My script would look at each existing deck, run through the full deck (verifying each card) and then verify the final deck count. If it passed my checks, the deck would convert, and if not the deck would simply be ignored. Once the script that completed, it converted roughly ~95% of the decks.

Unfortunately, things just never progressed from there. I ended up getting fairly busy IRL and the task ahead of me still seemed absolutely massive and it was getting hard to motivate myself for the many, many hours required for the overhaul. Things went quiet and work essentially stopped on the overhaul. It wasn't until June 2022 that I found myself with much more time and as such much more motivation to start up the overhaul project again. Feeling refreshed and envigored I dived right in and started making progress at a much faster rate. 

Fresh Coat of Paint

Next up I needed to cover the user accounts. This was by far the easiest conversion and I had the script done in essentially a day. User passwords had to be scrambled as I was using a stronger password storage system, than the old WordPress database I was using. This is essentially why users received an email asking them to reset their passwords once the conversion was complete which by the way was another cause for headaches! For those who know, native PHP isn't fantastic at sending a large number of emails and if you managed to send a large number of emails without tanking performance then there was a strong chance most of those emails would end up in the spam box. I needed to send over 120,000 passwords reset emails! After some research online I decided the best approach was a one-off usage of Mailchimp. Using Mailchimp, I was able to spin up a custom email with the required password instructions and easily send it out to 120,000+ users. It wasn't cheap however and I shed a tear as I had to drop $1000 to be able to send to that amount of users with the custom information I needed in the email. 

The lovely Deck Builder we have (crafted by the excellent Felix) also needed some adjustments. We needed to be able to extract the deck being built from the deck builder to allow the user easily and seamlessly submit the deck to the website. Felix was brilliant at this and had a working solution created in no time. This essentially wrapped up the small amount of work needed for the Deck Builder to be compatible with the new system. 

Articles and comments were next. Articles were pretty easy as it was essentially the same as converting the decks and they were stored nicely in the WordPress database. Comments however were another story. Remember how I was using PHPBB? Well, BridgeDD had a neat feature where if you commented on the forums (for a deck) it would synchronize over the main WordPress site and show that comment while viewing the deck (and vice-versa). This was known as cross-posting. Cross-posting worked really well in practice but made things exceptionally difficult when I needed to pull comments that were now spread across both WordPress and PHPBB databases during the conversion. Thankfully, BridgeDD makes some lovely cross-post database tables that allowed me to query when content was cross-posted. Unfortunately, this raised more issues. Comments that were initially posted on PHPBB were done using BBCode (a lightweight markup language commonly used in forums). I now needed to pull the comments and insert them into my new comment database and then run through each and every newly inserted comment and convert the BBCode found. While it wouldn't have been the end of the world to leave them as they were (there wasn't a huge amount of comments utilizing BBCode) I just felt like it would make things that little bit nicer if done.

Now that I had most of the content converted, it was time to actually start refining the site. This included porting over functionality from JavaScript functions, fixing any PHP errors found, and enhancing the overall speed of the site. I like to call this time Limiter Removal  since I felt like I was massively overloading myself to get to the finish line. 

 

Limiter Removal

Porting over most of the JavaScript functions was actually quite easy and I had the majority of the port work completed in a few days. The Pack Simulator, Collection Manager, and Deck functions all worked correctly which took up the majority of functions that needed to be ported.

At this stage, I was using PHP 7.4 on the new (test) site but I had a nagging feeling in the back of my head to go ahead and upgrade to the relatively new PHP 8.1. Performance didn't seem to differ much between PHP 7.4 and PHP 8.0 but there was a very nice bump in performance between PHP 7.4 and PHP 8.1. I didn't want history to repeat itself so I went ahead and upgraded to PHP 8.1 and yes, things broke.

I do jest as it wasn't actually that bad. The first thing I needed to do was get Memcached up and running for PHP 8.1. This was fairly easy and I had it completed in a couple of minutes. We are using Memcached on the new site to track information such as cards, decks, and article views in a very lightweight manner. 

PHPFastCache was the next item that needed to be "fixed". I was actually using a relatively old version of PHPFastCache on YGOPRODeck that was still compatible with PHP 5.X versions. Upgrading this was initially easy but they seemed to have made one big change with the latest major version of PHPFastCache with how you called the default cache configuration.

For example, this was how I was calling the old configuration:  

CacheManager::setDefaultConfig([ "path" => "my-cache-directory" ]);

And this was how the new configuration needed to be called:

CacheManager::setDefaultConfig(new ConfigurationOption([ "path" => "my-cache-directory" ]));

Naturally, this is a small change but I cannot being to explain to you how often I was using this piece of code across the site. Almost all of my APIs were using it and the majority of actual pages were also making use of it in one form or another.

Thankfully, some useful "find and replace" regex helped me update the majority of them while the rest needed to be converted by hand. Once this task was completed, I then needed to move on and fix the newly assigned PHP 8.1 warnings that were littered across the site. A PHP warning isn't the same as an error per se. A warning is more along the lines of "you should be coding this differently" essentially and usually for good reason. After a couple of days of trawling through the test site, I had almost all of the warnings fixed and cleaned up. The site was now PHP 8.1 compliant!

Elbow Grease

It was around this time that I had given myself a nasty case of bursitis. I had never even heard of bursitis before then! Bursitis occurs when your bursae become inflamed. Your bursae is a small fluid-filled sac on your elbow and let me tell you something, it hurts when inflamed! I knew immediately why it happened. I had been coding non-stop on the test site and had been inadvertently leaning hard on my elbow while coding. After a week of non-stop coding, I had a pain in my elbow that I didn't think much of. That night however the bursae in my elbow had ballooned to the size of a golf ball. That morning I visited my local A&E department and queue 6 tables a day every day for a week and it had returned to normal. I had to massively limit the amount of coding I was doing that week as putting any pressure on my elbow resulted in immediate pain. I suppose it is very similar to cards destroying themselves from the effects of Limiter Removal eh?

A New Era

Once I had recovered, it was time to jump back into action! There wasn't a huge amount left to do but I mainly spent the time testing the "user flow" of things. Making sure submitting/editing decks, post comments, forum threads, etc were all working as intended. I fully expected to have missed stuff and I'm eagerly awaiting the throng of bug reports that inevitably await me but I finally made it. Here we are! A new era for YGOPRODeck!

Some of you might be thinking, was it really such a big deal? To me it really was. YGOPRODeck was the site that I nurtured and adored over time. I really, really wanted to improve it. And I really hope this new site feels like an improvement to most of you! If anything, it allows the site to live much longer now since it can easily be maintained and improved. I don't have to fear that a PHP update or a WordPress plugin will tear down the site now.

I honestly hope you all enjoy the new site and I hope you have enjoyed this short little article! It turned into more of a "history of YGOPRODeck" than I had expected but I really wanted to get it down in writing to allow myself to look back on the work I have done and for once tell myself I did a good job. Something I've struggled with since leaving college! 

For all you Yu-Gi-Oh! fans out there. Enjoy!