How do Databases Really Work in a Web Application?

For the past few months, I have been exploring computer programming by solving simple math and logic problems. Writing code to solve for a prime number or to calculate the day of the week 300 years from now has been exciting, but in a lot of ways that kind of coding is one-dimensional. My latest project has been an endeavor in solving problems in a more abstract way. According to John Parman, UC Berkeley scholar and author of The Inevitable, new technology is most effective when it creates a platform upon which users can contribute content rather than simply consume it. Twitter, YouTube, Facebook, Airbnb and many other web applications actually produce none of their own content but are wildly successful, because they deliver personalized data in a convenient package. I say all of this, because my most recent web application is a small step towards a much larger idea that I have.

I aim to build a swiss army knife-like tool that can improve the lives of classroom teachers. The application, that I have just developed for that multi-tool, is a seating-chart generator that uses data to inform the way that teachers create effective teams within the classroom. Dr. Spencer Kagan, a pedagogy researcher developed these techniques, and I am attempting to automate some of the tedious number-crunching that teachers must spend valuable time doing. By developing this application, I gained a deeper understanding for a variety of programming techniques. I aim to break down the architecture of this web application step-by-step in the hopes that other aspiring developers may use this as a road-map for developing a web application from the ground up.

Process:

1. Outline the idea — you can’t build anything without vision.
2. Set up your working environment
3. Build a database and determine how that data is related.
4. Build the controller and decide how data will flow.
5. Create views to make that information visible.
6. Test
7. Test
8. Tell yourself that you did it
9. Test again
10. Launch!

I won’t spend any time talking about ideas for a couple of reasons; (1) I did so in my last blog and (2) dreams don’t come from a blog. I’m guessing you already have a dream if you made it this far, so let’s just jump into how to make that dream a reality.

For the record, I will use my application as a running example to illustrate some abstract points as we go. I recommend watching my application walk-through before reading further. I am also going to assume that you have a basic working knowledge of Ruby. This one is for all of those Flatiron programmers out there getting ready to start your phase two project.

Step #2 — Set up your Working Environment:

Take a look at my repo here and focus your attention on the file-structure, Gemfile, config.ru, and environment.rb. It is important to note that various gems do not play together nicely unless the correct version numbers are maintained. Spend some time reading appropriate documentation around your gems. This may take around 20 minutes, but it will save you a lot of headache. Also note that the config.ru file is the entry-point for the application. The .ru means rackup and shotgun is a developer shortcut that launches this file. In this application you will notice that it will throw an error if a database has not yet been created. If that database does exist, then it runs the ‘ApplicationController’ file, which brings me to the next major hurdle — data.

Step #3 — Build a database and determine how that data is related:

In this application, there are three major classes. Users are the most senior class and user objects are instantiated when a user signs up using this application. Sections and Students are subordinate classes but their relationship in this application is has_and_belongs_to_many. This relationship is less about hierarchy and more about how I reference the objects later. Instances of student objects have a section_id foreign-key attribute (column in the database), but there are multiple users who have a 1st period during their day. You could imagine that if this app became popular then thousands of teachers across the country would create ‘section_1’ instances of the section object. Each student only belongs to one section by that name. ‘Section_1’ is not unique and therefore not superior to any student. This relationship may seem odd, but when I reference a user, I need to be able to call all of the sections that belong to that user. When I reference a section, I need to be able to call all of the students that are associated with that particular section which is owned by the user in question. Here’s what I mean:

Sample Section Object

Sample Student Object

Edgar is a student in the database with a test score of 95. He is a student in user #1’s first period. The section needs to be associated with a user and it cannot have many students, because those students could be incorrectly assigned. Instead the data lives in the student. Instances of student objects know what section they belong to and what user they belong to as well. These columns in the database make the student object more dynamic and malleable as the application undergoes change. Building a database is about knowing what your objects should be able to report. Think about how you are going to call an object to determine what sorts of classes will be required. From here, you just need to build some objects, run your rake commands to get those tables created, and adjust your schema to account for the attributes you need. With a solid database, the next step is to instantiate objects using some forms!

Step #3 — Build the controller and decide how data will flow:

When a user submits a form, the idea is that your database needs to either retrieve some data or add some data to the database. This is the essence of getting and posting. If you need to see the database, you are ‘getting’ data. If you need to edit the database in any way, you are ‘posting’. The first post that your application will make is to post a new username and password. Review the ‘bcrypt’ gem documentation if you need to wrap your head around the details of this concept. Once your user ‘signs-up’ (aka: posts their username to the database), the next move is to ‘sign-in’ (get their username from the database). Users can then get down to the business of instantiating objects. In my application, that means creating instances of section objects. By filling out a form, users can input certain information, but other information is hidden. In the create-section form (which has a post method) users only input a section number, but the form also submits a user_id that is taken from the session object. Users can view sections, and within those sections they can instantiate students. The create-student form also includes some hidden input values to keep the data clean and organized. Users can and will click anything, so I kept as much hidden as possible. Once the data is in the database, displaying that is as simple as calling on the object and passing arguments that fit the query. Active-record, Sinatra, and ruby really play nicely with each other and make this whole process very smooth.

Step #4 — Create views to make that information visible:

This is the point in the project where I came to the realization that I prefer back-end programming work. Figuring out how data will fit together and communicate is a lot more exciting to me than figuring out where a button ought to go. My application is also evidence of that bias and I will have to attack this weakness going forward. Deciding how to structure urls does present some challenge and I did enjoy this process. Presenting less information in the url adds a layer of security and makes the user experience cleaner.

Step #? — Testing:

Testing is where I spent the bulk of my time. For reference, I built my application in about four days and I spent nearly two weeks testing. Thinking of ways that I could abuse my application or submit corrupt data is a lot like removing random screws under the hood of your car just to see what happens. To create a robust application it must be done, but the repair work takes time. In the end, I found a new love for class validation statements. These short lines of code in the model prevent nil or corrupt data from entering the database in the first place. This saves time! Once the database becomes corrupt, it appears to be completely inoperable (or at least I haven’t found a solution yet). Going forward, keeping that data clean is priority numero uno.