Watching videos instead of reading books

Lately I’ve been consuming a lot more video media than I used to. Until recently, most of the media I’ve consumed was by reading. I would either read non-fiction books, manga, or a lot of reddit. My average day would be me waking up and browsing reddit posts on my phone, setting aside some time during the day to read books, and then the latest manga chapters before going to bed.

Why I'll never get tired of Math

One of the earliest memories of my mother was when I was a kid and she would teach me math in the living room. I was just old enough to talk and walk around at the time. I remember that she would create a 10 by 10 grid on a piece of paper and have me fill it up with numbers - to count from 1 to 100. I would pull out a pencil and start writing out the numbers, saying out loud each number as I wrote it down.

It was actually pretty fun for me at the time. I liked using a pencil and writing out numbers, and I liked filling out the grid nicely. I remember having one of my first “aha” moments when I saw the pattern that the 20’s are just the 10’s except with a 2 instead of a 1. I liked looking through the grid and trying to find the cool numbers, like the palindromes or the big ones like 100.

I only have two memories from kindergarten - playing Cubix race’n’robots on Playstation 1, and the time the teacher asked the class what the number after 10 was. Although this isn’t an actual achievement, I thought it was cool that I was the only one that new it was 11, when the rest of the class was shouting out numbers like “a hundred!” and “a million!”.

Ever since then, I’ve always like math. Despite moving from India to America, and switching schools across India and Queens, Math was a constant that I felt good at no matter where I went. In elementary school I was always the one raising my hand to answer questions, and genuinely really liked class because I would learn new and exciting things like fractions or square roots.

Time progressed and math got a little bit more advanced. Towards the end of middle school I felt confident that I was good at math. I joined the math team at my school and as a team we made it to the state-wide level at Albany, NY. At the city/county level I did a good job, but participating in the state level really humbled me because I ended up doing the worst on my team. I still had a lot of fun though, and competed a couple more times in the Purple math contest and at a local competition in the bronx where I came first place.

In high school I wasn’t on the math team. At that point, the only time I competed was in the AMC/AIME competition. Now that I actually had friends, I mostly just stuck to taking AP classes in school, but not really doing much outside of school. I still excelled in math in school, and I liked proving theorems and learning new formulas.

My first year at SUNY Stony Brook was the first time in my life where I wasn’t taking a math class. I took intro classes to Computer science, Chemistry, and a little bit of chinese on the side. I was having a good time at school - learning things, making friends, and getting involved on campus.

Something just didn’t feel right to me though. I was learning so many things but I didn’t know any more about numbers. I knew more about computers but was starting to forget formulas about math. I had a feeling in my chest that something just wasn’t right. In my mind I had always thought learning and math were inextricable. How could I feel like I was learning when I was forgetting my math?

I wasn’t a math major at the time, but this feeling made me want to be. I spoke to my advisor and friends about if it would make sense to do an applied math double major. I wasn’t sure if it would give me enough value for the amount of work that it would add. I consulted some friends about it too. To sum up the advice I got: It would be a lot of work and it won’t help if you’re going into software engineering. I didn’t like the sound of that advice, but I knew it was right for me.

I continued as usual at Stony brook, but it still felt wrong. It would be a lot of work to get a double major and it wouldn’t help me in the future, so why would I do it? I started to feel overtime that the usefulness of the major mattered less and less to me. I just wanted to learn and do more math, I didn’t care how useful it was. Later that year I ended up declaring the double major and filling out the following semester with math classes. I just did the major for fun.

That moment - what I deliberately chose to learn math even though there was no point made me realize how much it mattered to me. I’m out of school now, but when I read books or watch Youtube Videos, I like to pick things that teach me anything new about math. Whether it’s statistics or number theory doesn’t really matter to me - just the process of learning is fun.

Lessons in leadership: My last year and a half at Google

For the last year and a half I’ve been working on the Cloud Logging Front end team in GCP at Google. The time I spend programming is mostly client side code using Angular/TypeScript. I also write middleware in Java between the client and the back end.

Google is definitely a place to find challenging work. I’ve encountered a lot of very difficult front end problems in my short time here. I don’t just mean that in a coding complexity sense, but also in a user experience and accessibility/internationalization sense. In college I always thought that the hard part of the job was writing the code, but that isn’t true at all - especially in the front end development.

The best way for me to sum up what I’ve learned about software development and leadership at Google is by me running through the roadmap of what it takes to launch a user-facing feature. Some of these things may be easier at other software companies, but some of them are probably also harder.

In the interest of not giving away what I do at work - I’ll use an example of a personal project I made - the Messenger bulk archiver. This project is just a chrome extension that goes through your chats on messenger and then archives them all for you automatically. It was fairly easy for me to develop using a little bit of javascript and a little bit of money to buy a license to sell software on the chrome store.

If I worked at Facebook, I might try and make a little button at the top of the screen that says “Archive all” which just archives the chats properly using the messenger back end. Here is a vastly oversimplified map of what I might have to do:

1. Speak to Product Managers to determine if this something customers want. Obtain consensus to experiment with this feature.
2. Work with UX to design some experimental mocks
3. Conduct User research with users on the mocks
4. Analyze research data to determine if this would help users, and how it might affect user happiness or some other metrics
5. If results are good, run the feature by legal to determine if archiving people’s chats in one button doesn’t breach any laws since the button involves marking the user’s chats non-manually
6. Consult Facebook security teams to determine if the button might allow users to become victim to some potentially harmful attack
7. Begin an engineering design and obtain approval for engineering stakeholders - this at least includes some messenger back end teams and your front end team.
8. Implement the feature and guard it by flag so that it can be rolled out incrementally
9. Roll out the feature to some set of users and then perform some analysis to determine how users are impact with or without the feature
10. If the feature does well, roll out to all users
11. Put measures in place to make sure that the feature is always online and that engineers are notified if it breaks, so that it can be rolled back immediately

Taking a look at this list - it actually looks like implementing the feature is the easiest part of the map. Among these 11 steps, a Front end engineer would likely be a part of at least 7 of the above steps - if not all of them.

It follows that half of the job of a Front end engineer is to know how to get people to work together and create consensus. The stakeholders might be Product managers who want to push new features or Back end engineers who want to do less work. Having smart and opinionated co-workers who are good at arguing makes it even harder if they disagree with you.

Two core skills that are important to get anything actually done for users are being able to talk in “corp-speak”, and being able to set up effective meetings.

Corp-speak is the way of talking at work in a way that gets things done, without directly assigning blame or doing anything that might cause unwarranted harm to any party. This entails speaking in a way that makes you sound confident so that people can trust you. It entails not calling out your coworkers for mistakes or miscommunications. But most importantly, it entails making it clear who needs to do what, and who is responsible for what.

There are a lot of subtleties with corp speak because no one actually will ever sit down and teach you it. It’s something you have to learn on your own. If you’re perceptive, you can pick up on how people do their corp-speak.

On my team, instead of ever saying the phrase “I don’t know”, I say “I don’t have context” and then redirect the person asking me a question to someone else. This is a great example because the former makes someone lose confident in you and it doesn’t produce an actionable outcome. The latter gives someone else a path to move forward, without making them lose confidence in you.

A couple of other examples of corp speak are to ask “Are you familiar with X?” instead of “Do you know X?”, and saying “Can we have X done by friday?” instead of “Get this done by friday?”.

The other skill is being able to organize effective meetings. On paper this is actually really straightforward. Don’t let conversations get derailed, only invite people who need to be there, have an agenda. It’s all standard stuff, but I see people screw it up a lot.

A mistake would be to think that a meeting is “more important” or “more effective” if there are more important people, but that’s not true. Plenty of ineffective meetings happen even with upper management, since they’re all spread so thin. Only bring in people who are focused on the task you are organizing the meeting for.

Also make sure that meetings stay on topic. Agendas help people stay on track and be more deliberate with there time. Don’t assume that agendas will fix problems themselves. Expect to be vocal and have to consistently direct focus on the task at hand, since lots of people means lots of ways to derail the conversation.

Those are the really important things I learned at work over the last year or so. The stuff that’s really core is the non-coding parts.

My favorite parts of Pittsburgh

I really didn’t like Pittsburgh when I first got here. I had no friends, and it was nothing like New York, where I spent most of my time in the US. I wasn’t used to not being able to take the subway anywhere. I wasn’t used to having amazing food around me all the time. I wasn’t used to the lack of diversity of the people around me every day. I wasn’t a fan of sports which is what most people in the city do for fun. Like most things in life though, I persisted.

The primary reason for me being here was work. I spent most of my time at work, and the rest of the time was at home juggling or eating. When I first got here, I did go to all of the “attractions” in pittsburgh - namely the Duquesne Incline and Point state park. Both of them are pretty good places to see, but they aren’t anything compared to what you can see in New york city. Pittsburgh really just didn’t compare at all.

As I spent more time here, and made a few more friends, I started to see the better side of the burgh. The first thing you notice is that there is a lot more nature around. You can actually go around for a run or on a walk on a sunday morning without being run over by people in a rush walking. You don’t have to walk at the edge of the sidewalks to avoid the huge piles of trash that places leave out which take a long time to get collected. You can actually say hi to anyone at a restaurant and for the most part expect a friendly response. People are generally just happier here and more willing to share that happiness with others.

That happiness started to penetrate into my daily life. I feel like I enjoy the littler things in life a lot more. Some days I just feel like walking to work to take in the breeze. I really appreciate the times I get to go out with friends to a local concert, or see a local night market in the next neighborhood down. Pittsburgh isn’t the biggest place you think of, but there are a lot of smaller scale events going on all the time in the city.

Every year I’ve gone to ReplayFx, a huge arcade and pinball convention. I go to check out the car show in schenley park. I even go to the Three Rivers Arts festival every year. When I first got here, this all seemed like lame events that would be happening on any old New york weekend - but they’re a big deal here. People from all over Pennsylvania bring in their families and they all have a good time. You can enjoy eating pierogies, fries in sandwiches, or pickles in Pittsburgh without ever having to think twice. It sounds weird, but as I write it it’s very liberating.

There’s a huge amount of certainty that you get in Pittsburgh that you just don’t get in New York City. If you know me well personally, you would know that I really resented coming to Pittsburgh in this last year. Sitting down and looking at my experiences in retrospect now though, the good parts were really good. At the end of the day, I’m really grateful to a city with objectively less to offer, that is still able to make me a happier person.

I Quit learning flow arts and juggling

I first started glowsticking in my junior year of high school. I joined the stuy rave club, now known as stuy flow. It took me a really long time to learn even the most basic tricks - the 2 beat and 3 beat weaves.

To give some context, the 3 beat weave is the go-to glowsticking move that is used as a transition between a LOT of glowsticking tricks, especially for beginners. It also requires you to spin the glowsticks in a sort of hard to understand pattern for beginners, and it often results in a lot of bruises. It took me 2 full weeks of hitting myself, and going to the club nearly every single day after school. At the time, it was the hardest test of raw will power and persistence in my life.

When I finally learned the move, I was ecstatic. All of the bruises felt like they were for something, and I had something to show to my friends for the experience I went through. I got to see the smiles of the people who had been with me teaching the move the entire time. It all felt like it was worth it.

I kept learning new riskier and fancier tricks after that. I practiced to perform at the school-wide shows with the other club members. I became a senior member and started teaching new members who came into the club. I became the director of the next show. I made more friends day by day within the club. I felt like there was a progression in my life not just with glowsticking, but with my life satisfaction. I had more friends and I had actual responsibilities which made me feel like I mattered.

After I graduated high school, I mostly fell off when it came to practicing glowsticking. Here and there I tried to teach classes in the city, make videos, or just spin fire poi with friends. For the most part, though, I focused on school. The compute science coursework I had in school became the new task I had to focus my persistence and energy on. Glowsticking became more of a stress relieving hobby for me. I did also perform at a couple of talent shows to raise money for UNICEF as well, so it felt like I was using it to give back to the community.

I graduated in 2018 and have been trying to keep up my glowsticking and flow arts. I’ve transitioned into doing less glowsticking and more poi spinning ( I don’t need to be as flashy, since it’s just for fun ). I bought triple staffs, attended flow festivals like fluggleburgh and kinetic fire. I actually got to meet a lot of people in the community, and I’d like to think I made a little bit of a name for myself teaching a class at fluggleburgh and teaching classes at Carnegie mellon university. I also briefly attempted to start up a juggling club at my office in Google Pittsburgh. Flow arts somehow never managed to leave me after all of these years, and my props are sitting in my closet ready to be picked up today.

But today, I just don’t feel like learning it anymore. I just wrote out a brief history of my experience with flow arts, and chances are if you’re reading this you’ve seen me perform at some point. Maybe you think I’m insanely good and flashy, maybe you think I’m an amateur, or maybe you’re wondering why I even got into this in the first place. The point of this all is that most people only ever get to see the good side of my experience.

What people don’t see is me practicing in my home or in my room. I attempted to show off this side of myself on instagram for a while. I consistently would post videos of me juggling or participating in challenges. It was meant to give people an insight into what it’s like to practice every day for something. I hoped it would give people a glimpse of the hard part of it, and would hopefully inspire other people to pick up the challenge. I even saw a small growth in my instagram following from all of the videos I posted publicly.

At a glance, it might seem like this all is working as intended. I’m gaining a following, and my friends get to see me practicing so they appreciate it all, right? Not so.

There were a lot of stressful unintended consequences of me posting these videos online. First off, when I started posting these videos, I actually had already accumulated a decently sized number of new tricks that I could show off. I was able to sustain putting out new videos and learning new things pretty well. Things started to break down when I was pushing up against the limit of how fast I could learn things.

For reference, in order to master 3 ball juggling, it takes the average person somewhere around 15-20 hours. I posted one video where I made a montage of me juggling clubs. In order to juggle 3 clubs consistently, expect to take another 20 hours. This meant that every waking moment I had outside of work, I was in between juggling learning sessions at home or at the CMU juggling club. Admittedly, this was pretty fun for a while.

After a while though, learning juggling becomes incredibly frustrating. Imagine if you had to try some hundreds of times to throw a bunch of things in the air and catch them, and every time you failed, you had to pick them up, focus, and try again. Now imagine doing that for 20+ hours to learn the basic 3 object juggling pattern.

The problem is that it is just not worth it for the amount of time and frustration you have to put in. The juggling example is relatable because at some point, most people have at least tried to juggle. For other more advanced flow arts, it was actually a lot worse that 20 hours because the moves just get more and more nuanced. There are fewer people to teach you, and you have to often go out of your way to festivals or to meetups just to get a routine that will maybe eventually guide you to the trick. This means sacrificing your own personal time with friends, or taking time off of work. Not to mention you don’t have any metric of if you’re doing it right or how long it will take you to finally learn the trick.

I still spin poi and juggle occasionally, but I quit actively sitting in my room trying to learn crazier and crazier tricks. This doesn’t mean I don’t appreciate the art form as much as I used to - I still watch videos all the time and follow a bunch of spinners on instagram. It just means I want to take the time from learning those would-be moves and put the time and energy into focusing on new hobbies like writing blog posts.

How important is coding fast?

Every once in a while I find myself in a conversation with a friend about typing. Sometimes they talk about their typing speeds, sometimes about the text editors they use, and sometimes even the keyboards they use. As someone who has to type code for a living, I decided to see what I could do to optimize my programming before I start my new job. I already knew that increasing my actually typing speed with practice wasn’t going to help too much. The best thing to do was to learn a new text editor with a lot of shortcuts and tricks to make things easier. I wasn’t going to bother buying a fancy new keyboard because that would cost too much time to learn, and possibly a lot of money. Then I started to think - what is typing really and what’s the best thing I can do.

Logging machines

Typing is, first and foremost, just a method of logging down your thoughts. Handwriting and typing have the same purpose, but typing just makes your thoughts easier to spread and process because it produces digital logs. What we’re interested in isn’t necessarily typing fast - we care more about logging fast. To generalize, think of editors as just a type of logging machine - a tool that takes inputs (keystrokes) and outputs logs.

Logging machines don’t have to be editors like Microsoft word, Emacs, or notepad. A pen and paper are a logging machines with handstrokes as inputs. A camera is a logging machine with a visible light input. Even the copy/paste tool is a logging machine that takes highlightable text as input. It happens to be that some machines are better than others at logging down certain thoughts - it would take me a very long time to log the contents of a 300-page book to another book with notepad, but I could do it really fast with a printing press. A nice thing about logging machines is that you can build one thats tailored to the thoughts you want to log. It (mostly) does not matter what the thought is.

It follows that in theory, a machine can only ever log as fast as you can think. Keeping that bottleneck in mind, the theoretical optimal editor doesn’t just let you type inputs as fast as you can think - it also interrupts your thinking the least. Additionally, the optimal editor should not take any time to learn and should be completely intuitive. Basically, we want to come as close as possible to a machine that literally hooks up to your brain and puts your thoughts down. While that’s probably never going to happen, it is interesting to think about all of the properties of logging machines to better compare them.

Things to consider

• Input Complexity - An editor with a lot of inputs is going to not just require a lot of learning time, but it might also interrupt your thinking while typing. For example, notepad has few inputs - just the letters on the keyboard plus a little more. Vim on the otherhand uses a lot of keys you wouldn’t normally use like ESC and CTRL, and basically has a different function for every combination of keystrokes imaginable. We would like to minimize the number of inputs as much as possible.
• Input tools - It should be easy to input into a logging machine. It takes a lot of time to use a mouse (think of the last time you used a visual keyboard on your PC), so we want to minimize the time we spend switching between mouse and keyboard.
• Generality - If i’m writing a document, my editor needs to be versatile enough to handle and maybe input images. The ability to handle images will require additional input complexity, but it will greatly add to logging power.
• Similarity to known machines - It’s generally a lot easier to learn an editor that is similar something you already know. For example, you should be able to pick up Google docs pretty fast if you already know Microsoft word.

My final choice(s): Vim and Sublime

I think that because different editors are better at doing different things, its natural that I select two different editors for the two different kinds of programming I do.

For java, python, and scala programming I use vim. Despite the fact that it’s complex, it allows you do things all on the keyboard. No more clicking around to navigate and open files. Vim is also great because it’s customizable, so I can easily add shortcuts from other editors that I like - such as the ability to move lines up and down. Also, vim runs in a terminal which is very convenient for any sort of writing on a mac or linux. For anyone interested, by .vimrc is here..

colorscheme slate
syntax enable "Enable syntax highlighting"
set tabstop=2 "Tabs are displayed as 2 spaces"
set softtabstop=2 "Tab actually enters 2 spaces"
set expandtab
set number "Display line numbers"
set wildmenu "Tab completion for files and commands"
set lazyredraw "Update display only when necessary"
set showmatch
set hlsearch "Highlight all matches when ? or / searching"
nnoremap J 0v$yddp
nnoremap K k0v$yddpk
nnoremap <leader><space> :nohlsearch<CR>

For HTML, CSS, and Javascript I like to use sublime. The convenient thing about sublime is that my mouse is already going to be moving around because I’m testing out website front ends, so I don’t feel slowed down by using it, and I get the benefits of it like easy copy/pasting. Sublime also has a great plugin for HTML writing called emmet which allows me to create tabs insanely fast without ever having to type a ‘<’,’>’, or ‘/’.

I highly recommend both vim and sublime for productivity. Even though I probably wasted more time researching and learning editors than I will save, I recommend everyone to go out and try to learn something new. You might find something new you like.

Books I read in the last year

In the last year I actually ended up reading a few books (no, I don’t know how I did it either). Here I’m going to list out some important take away points from each book and what I think about the book overall. All of the books I read were non-fiction and somehow statistics related so they actually did have “lessons” in them. Here they go in order:

The Signal and the Noise - Nate Silver

This book has a good premise. Whenever humans try to make predictions or forecasts (they are different) they have to base them on some information. The difficult part of this information is that a lot of it is “noise” that doesn’t matter or just confuses us. What we need to look for is the “signal”.

The author makes a lot of cool case studies, such as how we have no idea how to accurately predict earthquakes but people try to (mistaking noise for signals). It also brings up the financial crisis and how many companies including the ratings agencies like moody’s saw it coming but chose to ignore it (ignoring the signal). It also brings up a cool story about how world champion Garry Kasparov lost in chess to IBM’s Deep Blue AI. He then talks about statistics for a while.

The author’s writing is good too but my problem with the book is that it doesn’t make clear how to tell signals apart from noise. In fact, all of the case studies make the comparison in hindsight. For me its a 3/5. I recommend it only if you are interested in this sort of stuff abd don’t know too much about it already.

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy - Cathy O Neil

You would never think that big data and machine learning could actually have a negative impact on parts of society. That’s the big idea that this book explores. The case studies that are presented in this book are extremely eye opening and they highlight the problems of AI and machine learning in our society.

If an algorithm is based purely on a lot of data and math, it is probably right is one of the core ideas. The author notes that while machines help our society operate more efficiently, the businessmen that use the machines usually have absolutely no idea when they work. When people come to them to question the algorithm, they can’t change anything about the algorithms and the easy reply is to trust the algorithm. It can lead to a vicious cycle.

One eye opening case study the author points out is how targeted ads are used to pitch ads for for-profit colleges to poor neighborhoods. These colleges are about business and the degrees have been shown to be not worth much at all. Yet the targeted ads keep students going to them, and keeps the people in those neighborhoods in a vicious cycle of poverty. Another case study was with a crime prediction software used by the NYPD. Engineers fed decades of crime data into the machine learning algorithm to make it predict where the next crimes would most likely occur. It turns out this software keeps police patrolling the same low-income neighborhoods mentioned earlier. Moreoever since they patrol those neighborhoods more, they get more crimes there and the machine keeps telling them to go there. The worst part is no one can question a machine.

I highly recommend this one especially if you interested in ethics and big data. The writing was decent, the thing about this book that will keep you reading is the case studies. I give it a 3.5/5

What the Luck? The Surprising Role of Chance in Our Everyday Lives - Gary Smith

Despite being the book that taught me the least among the rest, I actually liked this one the most. The writing is very good and the way the author explains stuff makes a lot of sense.

The big thing that the author talks about here is called “Regression to the mean” and he gives a lot of cool examples. My favorite one is this: Smart women tend to date guys who are not as smart as them. It turns out that that is a fact. Now it’s easy to say that women like to feel intellectually superior or that guys are just not as smart in general but the author makes a key point. If you are a smart person, you are simply less likely to meet someone as smart as you. Everything in the book is based on this. He also talks about some other interesting things like standardized test grading and air force pilot training, but you can check it out yourself.

I definitely recommend this book, its worth a read if you want to learn more about how randomness plays a role in your everyday life. I give it a 4/5.

The Black Swan: The Impact of the Highly Improbable - Nassim Taleb

This book probably had the most knowledge packed into it, but it also had some of the most pretentious and hard to understand writing in it as well.

There is too much in this book to unpack but I’ll explain the title. It turns out that back in the day during the age of exploration people believed that all swans were white. They thought so cause they only ever saw white swans. Until the one day that they did see a black swan and it challenged everything they thought they know. This book is about those “black swan” events that no one sees coming. The author characterizes them, describes when they show up, and what kind of impact they could have.

If you’re looking for a good read don’t bother with this. If really you want to gain some knowledge about how uncertainty takes form and shapes events in the world, check this book out. I give it a 3.5/5.

My most useful git tips and tricks

Most people see git as a nuisance when they first learn about it in school. There are a lot of commands to learn. Fetch, clone, push, commit, add, reset, stash, and so on. What I did when I first started is just learned the bare minimum of the commands I needed to know. Clone to download the repo, add to add changes, commit to persist changes, and push to push changes to the repository on github. It became a cycle -

(1) git add .

(2) git commit -m “Brief commit message”

and occasionally,

(3) git pull

After all, for most projects (especially solo ones) that’s all you need. The truth is each one of these has caused me a lot of problems before. In this post i’m going to outline the problem and then put a better solution to use for each one. Trust me, it’ll save you a lot of time and headache if you follow this post.

git add .

The problem with this command is that it adds every change to your list of staged changes. Isn’t that what you always want anyway? Not always. When it comes to group work this single command can waste hours of time if one person accidentally pushes a set of cached files or IDE settings. For example, if I push my eclipse settings which link to JRE8 and my coworker who pulls it has JRE9, it will break their program. It’ll also take a while for them to find the error because the commit message will say something like “Added this feature to this module”, but it’s neither the module nor the feature that caused him the error in reality. The only giveaway that it’s a setting will be that the code works on your computer but not theirs.

How can we get around this? You might think to just add the settings folders to your .gitignore but even then you might stage accidental changes. The only good way to do this is to take your time to manually add the files you want to add. git status will tell you a list of unstaged changes and you can just go through them and add them individually. This may seem like a lot of work, but if you are commiting in proper increments you should only be adding a few files at once.

git commit -m

This one seems like a really good command at first. You get to input a brief but descriptive commit message, commit all the changes you staged, and do it easily in one command. The problem is that the commit message that git automatically generates for you is probably better. It includes all the files that were changed (for debugging) and all you have to do is get rid of the ‘#’s to uncomment them

You can combine this one with the add in the last one using a command thats in the more updated versions of git called git commit -p. This great command actually automatically loops through every file change and asks if you want to add and commit it or not. If you master it and the shortcut, you can add and commit every file change in seconds, and without making the mistakes I mentioned earlier.

git pull

The problem with git pull when you work in a team is that it usually will try to do a merge and then force you to resolve conflicts yourself. 99% of the time what you really would like it to do is pull their changes and then put your changes on top of them (usually will avoid merge conflicts) and is what you want. You can accomplish this by using git fetch to pull the changes on the remote branches. Then, you can git rebase to move your changes on top of their branch, avoid marge problems and keeping your tree clean of MERGE commit messages. This will also make is so you don’t accidentally resolve the merge conflicts incorrectly and introduce more problems.

There you have it. A few new git commands that will help you improve your productivity in development. If you thought any of this was helpful, or if you know any better commands, please feel free to shoot me an e-mail or message.

How to learn coding without losing motivation

Sometimes my friends ask me, “How long would it take for me to learn python?” or “Where do I start coding to get a job?”. I tell them that if they pick a language and stick to it, then maybe work on a project or two, they will be well on their way to being a capable programmer. Then, they might ask me some follow-ups like “How exactly do I start? What should I download? Where are tutorials?” and I will point them in the right direction. I then go about my life as usual and then check on them a week later. What happens?

They’ve given up! I’m so confused! How could it be that my friend was so excited to learn about programming but quit so quickly? The process of obtaining the software wasn’t hard (These days, sites like codecademy let you type code on the web), the beginner code couldn’t have been too hard either, so what went wrong? I ask them what went wrong and usually it’s something along the lines of “I didn’t have enough motivation” or “I just stopped after a day”. I stop and think about it for a while.

Sure, it makes sense. My friends wanted to learn, but after a shoft while lost motivation even after they started. But this brings up an interesting question. How is it possible for you to learn how to code on your own if you keep losing motivation? The reality is that there is no way to magically keep a steady supply of motivation for you to learn, so what can we do? You hear stories about 16-year old geniuses starting companies with code they wrote from when they were accepted to MIT at 13 because they learned at age 9. But where do those people start?

To answer this question, we first need to look at some of the other underlying reasons why it’s hard to code without motivation.

It isn't fun

Compare coding to a hobby such as basketball. How come there are people so excited to go out to the courts to play and learn to play ball or watch it on TV? The obvious difference is that basketball is (for many people) actually a lot of fun even if you are just learning. You can still play with other people who are just starting out, and you will have a good time. When you code, it’s typically alone in front of an IDE or some tutorial. When you remove that fun aspect, your incentive to learn drops dramatically. This is no surprise, but what can we do about it? Well some sites like CodeFights have tried to fix this by turning coding into a game, but generally this won’t work. One possible way to fix this is to get a group of friends to learn with you. The addition of other people could add a competitive element to your learning which might make it more enjoyable. However, don’t let another person’s success demotivate you. This leads me to my next point.

Failure is unavoidable

I understood programming very quickly and was able to get my first project full up and running in less than an hour thanks to the tutorial - said no one ever. One thing that all beginners have to understand is that there is literally no one who thinks programming a project is easy. Ask any professional with years of experience and they will tell you just how much they struggled. I myself took 2 hours just to get my first “Hello World” program up and running because the tutorial didn’t tell me how to fix the compiler errors. When I was making this blog none of the posts would show up because the dates were in the future. When I made my first website I didn’t realize you had to link your css file in your html header and wondered why nothing was working. And all of this is not even the tip of the iceberg. If you are in a group of programmers and they talk about some complicated project they are working on, you cannot let this demotivate you a single bit. Instead, think about how much they failed to get to where they are, and realize that you can do it too no matter how many road blocks you face in the moment.

Lack of (Immediate) reward

There is a pretty big gap in what a programmer does when he is a professional, and what he does when he is a beginner. Many beginners have to realize that the process of learning cannot happen over a week, a month, or even a year. You might want to get your big automated mobile app website function compiler running as soon as possible, but you can’t do it without learning what a for loop is. Typically, once you learn how to print “Hello World”, you are not impressed. In basketball it is fun to make your first shot slam against the backboard and whoosh into the net. In programming, you are greeted with text on a screen. If you are fast, you might be greeted with an image on the screen in a few days. The point is, you aren’t really immediately rewarded for your coding efforts, and your brain sees that and decides to become demotivated. After all, why would you do something if there is no positive reinforcement in it for you? An easy solution to this is to simply reward yourself everytime you hit a milestone. When you learn the basics of a language go out for dinner, when you understand how pointers work go ice skating, when you make your first website go get a massage. It’s a simple idea that can go a long way in keeping you focused.

Start a project

A beginner might be tempted to do another tutorial after they finished the first one if they do not feel like they learned enough. To this I say, no. As soon as you finish your tutorials on a subject, I say you should jump right into a project. The main reason is that once you’ve started on something, you will typically feel uneasy if you let it go unfinished. As a result, you have a much bigger incentive to return to coding once you’ve lost your motivation. This will also teach you good future habits such as googling/using stack overflow if you don’t know how to solve a problem. In the end, you will eventually finish you project and give you something to be proud of.

Keep a portfolio

It’s a good idea to keep a portfolio of all projects you’ve ever completed because it becomes a great go-to source of motivation. To this day I feel an urge to program every time I look at my github repositories. It lets me look back at my work and say “Wow, I remember when I solved X with Y. I should’ve used Z instead”. It also gives you a more tangible reward for all of the work you put into learning.

Spend time with other programmers

If you want to learn programming, surround yourself with it. In fact, it is crucial in learning anything to surround yourself with people who are way better than you. When you’re surrounded with programmers and you really want to learn, you will see it for yourself. You will observe the way they solve problems, the way they stick to their work through hardship, the way they read documentation, and even the way they carry themselves in their daily life. When surrounded by people smarter than you, you can only get better. To this day I still look around me and see people who are better, and it makes me want to strive to become the best programmer I can be. It’s hard at times being the worst at something in the room, but find happiness in the fact that there is no where to go but up if you do this.

At the end of the day, you can have all the motivation in the world and still not be able to learn to code. At times like this its important to check with yourself: Why do I really want to learn to code? Do I even want to learn that badly? Because you will never be able to learn to do something if you don’t genuinely want to do it.

Transfer Protocols

In a world where the internet is becoming an increasingly integral part of our lives, the need for secure information transfer is greater than ever. On the web, information is transferred mainly through two protocols – HTTP (HyperText Transfer Protocol) and HTTPS (Secure). You have definitely seen both of these in your browser URL box, but what do they actually mean and how do they work? To answer those questions, we have to go into detail.

Transfer Protocols

The concept of a transfer protocol is simple. How can the client and server transfer information securely. By secure, I mean that transfer must ideally:

• Not be able to be intercepted
• Only be understood by the proper parties
• Not require prior contact between the parties

You can see secure transfers outside of the internet in many places. For example, the US postal service mails items in trucks so that other people can’t just steal your mail. Additionally, the only thing that the mailer’s must know about eachother is their addresses. They don’t have to agree on a envelope size, color, or even the postal service to use.

This isn’t anything new, Julius Caesar also used to send out letters to his allies. However, they didn’t have armored trucks back then, so he settled for a very primitive form of encryption known today as the Caesar cipher. Basically, he would “encrypt” his messages by shifting the letters of the alphabet a certain number of times. This way, even if an attacker intercepted the letter,they wouldn’t be able to decrypt it and understand it. Sounds good right? Almost. The only problem with this is that Caesar would have to communicate the number of times to shift the alphabet to his allies before the letter is sent out. It sucks for Caesar, but this isn’t a problem in the modern day. HTTP and HTTPS are vastly more advanced, but at the core they follow the exact same concepts.

HTTP

HTTP is a protocol that specifies the format of the HTTP request, which allows for information to be sent between client and server. HTTP requests are typically sent over port 80, but can be done over any port.

The format is as follows:

1. “HTTP://”
2. The url of the host domain (EX: www.hkattelu.github.io)
3. The path to the resource (EX: /blog)
4. Query string to search (EX: ?date=9.02.16)

This tells the server to return the resource at the path /blog that fulfills the query string’s requirement. If there is no resource or query string, the server will typically return the index.html file to client. The query string can also specify a POST or DELETE request which is meant to edit the resources of the server.

HTTP provides a solid interface for fast client-server interaction, but there is a problem. It is entirely possible for a middle man to intercept the requests and edit them for their own purposes. This can lead to bad things if the request contained sensitive information such as passwords, credit card information, or even a social security number. The attacker could also edit the request to attack the information of the server with a POST or DELETE request.

HTTPS

HTTPS is almost identical to HTTP from the user’s perspective, but there is much more going on that is abstracted from the user. There are only three perceived differences are that the URL contains an “HTTPS” instead of “HTTP”, that the request usually takes longer to process, and that the HTTPS URLS use port 443 instead of port 80 by default. Under the hood, HTTPS is a very secure transfer that makes it nearly impossible for an attacker to target a connection. The process is as follows:

1. The client and server agree on an encryption algorithm and SSL (Secure Socket Layer) version
2. The server sends a certificate to a trusted 3rd party to identify itself and obtain a certificate public key
3. The client uses its public key to encrypt the message and sends it to the server
4. The server uses its private key to decrypt the message and understand it
5. Both client and server now know eachothers keys and can send further messages securely

This is somewhat oversimplified, so I’ll explain more.

In the first step, the SSL is an interface that allow for encrypted communications through sockets. It is the link that messages will go through.

The second step is essentially a verification step. The server must send a certificate to a 3rd party to verify that it is legit. The certificate will usually include the details of the website like domain name, and the business registry info. It also includes a digital signature which truly verifies it. For example, you can claim to be chase bank to steal credit card information, but you cannot verify yourself without the digital signature. After the verification, the public key is obtained

In the following steps, there is an exchange of “keys”. Basically, a public key is the key that you share with others and you use it to encrypt data. The private key is the one you keep to yourself to decrypt incoming messages. The benefit of this two key system is that even if a middle man intercepts your messages, he cannot decrypt them or make any sense of them without the private key (which is never shared). This allows for a secure communication between only the client and server.

Keep in mind that although this HTTPS is vastly more secure, it comes at the cost of time. In most cases the time for the transactions to occur is negligible, but it is still worth mentioning.

Those are the basics of internet transfer protocols. I hoped this post helped you learn something new or clarify any existing problems.

My top 5 places to go in NYC

When you’ve lived in queens for years and went to high school in manhattan you eventually run out of things to do, even in the big apple. Sure, new stores open and new trendy hangout spots pop up, but for the most part it’s the same thing. It’s not a bad thing, but I always end up asking myself if it’s worth it anymore to visit Vanessa’s in chinatown or Gong Cha in midtown. Despite this, there are a few spots in NYC that I never get tired of visiting. They’re almost always fun or beautiful, no matter how many times I go. Here they are (in no particular order):

Washington Square Park

I come here often to visit friends at NYU or just to hang out with the NYU circus arts club. Even in the winter this place is always bustling with people. The fountain, the arch, the statues, and even the “pigeon man” all make this spot memorable. There’s always something going on whether it be yoga in the grass, musicians playing on their instruments, or the chess players in the corners. On top of being a beautiful spot overall, it’s also close by other notable spots like the Uncommons (board game cafe).

Battery park

The best part about battery park is that it’s by the hudson river. I used to come here after school to play frisbee with friends. Despite being on this list, it really is just a park. Most of the people here are either kids who just got out of school or parent’s taking their kids out on walks. The typical scenery consists of boats sailing across the river, people lying down to tan themselves, and guys playing handball and basketball. I don’t make it sound like much, but my writing can’t do the beauty of this place justice. On top of it all, this place is riddled with statues, has a movie theater nearby, lots of places to eat, and there’s a free kayaking spot at Pier 26 nearby.

Central Park

Arguably the most famous place on this list, Central park’s beauty comes from its sheer size. I’ve been here about ten times, but still don’t think I’ve even come close to seeing all that it has to offer. To put it plainly, every visit to central park is a completely new experience. The only downside I can think of is that it can take a long time to get out of here, so make sure you bring food and water.

The Highline

The Highline is an old and defunct train track turned into a park above the streets. In theory, you could see everything it has to offer in one straight walk. But the truth is that it’s such an active place that no two visits are really the same. There are many events for all ages held here, and tons of nearby restaurants in bustling downtown manhattan. But the true beauty of the highline doesn’t come from the place itself – but rather its surroundings. Walking the highline gives you a novel perspective of the city and its buildings unlike anything else. The buildings, billboards, and graffiti are what truly make this place shine.

Roosevelt Island

Roosevelt island, put simply, is one of the nicest places in the city. Compared to the other places on this list, it has almost nothing. It is simply a small island with a lot of parks. There are a few sites to see like the gondola to manhattan and the stone engravings at the end of island. For the most part though, it is just a really nice place surrounded by water. In a city where vehicle noise is almost everywhere, the serene sounds of the water beating against the rocks makes this one of my favorite places to be.

Those are my top 5 spots in NYC. If I had to give any honorable mentions, they would go to Governor’s Island (great place to bike, but it’s only open in the summer), and Brooklyn Bridge (nighttime walks on the bridge are arguably the most beautiful thing in the city, and that’s not even mentioning the DUMBO area). The next time you want to hang out with some friends and need a place to go, look no further. If you’re just a visitor in the city, make sure to check all of these places out. You won’t be disappointed.

Should you go to a hackathon?

About twice a semester I’ll go to a hackathon as a break from school to visit a new place, make some new friends, and see some familiar faces. What? Aren’t hackathons for making projects? Aren’t you going to be busy coding non-stop for 24 hours? Don’t you want to win a prize or build the next big app? The answer is a resounding no.

I’ve heard of a few myths about hackathons from friends and around the internet in the past couple of months, and a couple of them, in my opinion, are true. The majority of them are not. I’m going to get right into it and address three big ones in the following:

You need to build a project

This is the most common one and also the least true. I’ve been to seven hackathons since I first got to college and I didn’t even know how to declare a variable at the first two. For the next five, I either didn’t make anything or made a dirty mash up of code which I didn’t feel proud of. Does that mean I shouldn’t have gone to those hackathons? Not at all. In fact, I don’t think I would even be writing this blog post if I didn’t go to those.

The biggest reason would be that I just would not have learned as much. You should have two goals in mind when going to a hackathon:

• To learn something new
• To meet people
• To have a good time

But isn’t the whole point of a hackathon to build a project? No, let me explain. I can almost guarantee that at your first hackathon you will not be able to make a project or even come close. The basic programming skills that schools teach are good and are necessary fundamentals, but they will not teach you enough to make an award-winning project. Instead, you should focus on learning a new technology whether it be a framework or a language. You could do this by asking people, or by making a mini project, but there is no better environment for learning than being surrounded by people who are smarter than you.

For the same reason, you should be meeting people because they will (hopefully) inspire you. For me, seeing people who were so much smarter than me made me want to learn more. On top of that, you get to meet new and interesting people who will make you feel more at home in the hackathon scene. If you attend a hackathon away from your schools, you should try to have a good time so that you don’t regret spending your weekend away from home. You should only be trying to build a project if you are confident in your abilities and feel like you have a strong team. In that case, you can focus on that project instead of learning new stuff (don’t forget to sleep though).

You need a team

It can be a lot of fun to go to a hackathon and code with your friends. If you can’t gather enough friends, then going to a hackathon alone isn’t necessarily a bad thing. Every single hackathon I’ve been to has had a team-building period where you can either start a team on your own or join an existing one. You will probably meet a good team, but even if you don’t it isn’t a big deal. You could just leave if it is too much of a hassle for you, and spend the time working on your own thing. It might seem like a rude thing to do, but the team can find another person. I had a bad experience forming a team of random people once and I only wish I had left the team sooner. Remember, you should be having a good time.

Hackathons are competitive

Admittedly, there are a few hackathon-goers that are fighting to be the next Zuckerburg. The vast majority of them, though, are not. You can go up to any team at a hackathon and ask them for help and I can say with almost 100% certainty that they will go out of their way to help you. The hackathon community is not only really helpful towards newbies, but it’s also full of extremely intelligent and experienced people who have faced struggles of their own. Sometimes they can be weird, sometimes they can be narcissists, and sometimes they can be frustrated, but they will almost always be helpful.

So What?

I personally think that you should give hackathons at least two tries: one where you go without a full team, and one where you go with a full time. If you didn’t enjoy either of those, then maybe hackathons aren’t for you. For everyone else, I wholeheartedly encourage you to step out of your comfort zone and sign up for a hackathon as soon as possible, even if you know nothing about coding. You’ll have fun and hopefully gain some coding experience under your belt as well.

Why I study computer science

When I first started applying to colleges in late 2013, I had no idea what I wanted to major in. It was a big decision, for most people their college major was what they ended up doing for the rest of their life. I wasn’t ready to make that decision (and I’m still not) so I spent a lot of time thinking. People always say to do what you’re good at, and I was good at one thing – Math.

That narrowed down the options. I was considering some sort of science or engineering degree. Thinking about the types of classes I was taking at the time, I narrowed the results down even further. After considering the job market, I narrowed it all down to one option : Electrical Engineering.

Electrical Engineering? But you’re a computer science major! Hold on, the story isn’t over yet. While I was pretty set on electrical engineering I was also considering applied math or even physics. At the time, I had only ever coded in a functional programming language called scheme in a mandatory CS class my sophomore year of high school. I had little to no interest in it, especially considering my lack of experience.

My idea of computer science at the time :

I ended up choosing Stony brook university, a school known for its computer science program. I was still set on electrical engineering, but what changed my mind was my acceptance into an honors program for computer science. I decided that I would try it out for a semester, and if I didn’t like it then I would switch to electrical.

You can tell by now that I ended up liking it, but why? As I took classes I slowly began to realize that CS wasn’t really like math in the way that many people make it out to be. Sure, a lot of data structures and algorithms are based on mathematical principles, but what many people in the industry do is unrelated to that. There is enough software around that developers don’t really have to think that much about what’s going on underneath the computer’s hood. Usually the most someone will have to do is implement their own data structure based on an existing one.

That doesn’t mean the whole computer science field is mundane though. I just think that “software engineering” is a more apt name for the degree because most classes focus on the tools and thinking necessary to build efficient software. Don’t get me wrong, I do like building software with code, but it’s not what attracted me to the field. What I really like about CS is the problem solving aspect. Whether it be learning a new technology or getting my algorithms to be more efficient, I always have to think deeply.

Anyone who has ever made something with code knows that obstacles arrive more often than they’d like to admit. Overcoming those obstacles gives me a sense of achievement that I don’t think you can find in any other field. It’s tiring at times to always be wrestling with the compiler or the layout engine, but the rush of figuring it all out makes the effort worth it.

How I deal with procrastination

Whether or not you’re willing to admit it, procrastination affects all of us. Some people brag about it, most people resent it, but everyone wants to control it. I think that you can’t really control procrastination. It’s a natural behavior that’s innate to all people, but at the same time why do people have it? One could argue that it’s an evolutionary behavior. It encouraged early humans to stay in their comfort zones and away from predatory threats. It could be overcome by natural problems such as hunger, so that humans could go out and search for food. But in the modern day where all of our natural needs are more or less met, procrastination just hinders us from being more productive.

I’m not the worst procrastinator, but it can get bad at times. A lot of the time I can get started on a project and work on it non-stop for several hours, but getting to that point is hard. Even as I write this blog post I find myself constantly switching tabs to facebook messenger to read messages that I’ve already seen, checking my phone for new snapchat stories, and refreshing my e-mail boxes hoping for something new. Even though I know nothing new is coming, I still do it anyway.

Over my two years at college, I’ve come up with some good tools that help me battle procrastination. They don’t work all the time but sometimes they give me the push I need. Here they are:

Start small

Whenever I look at a new daunting project or homework assignment, my first instinct is to put it off until later. To combat this, I also break the assignment down into small parts, really small parts. Small enough parts that they’re easy to accomplish, but also give you a sense of achievement when you’re done. For example, the first step for me to finish up any of my AMS homework is to take out a piece of paper and write my name on it. For a programming project, this could be as simple as setting up a repository. Then I would follow up by creating empty files that I might need (Makefile, README, project.c), and then write up a few function signatures. On their own, each of the tasks is simple. When you put them together, you can get a substantial amount of work done without even realizing it.

Create a list

This one comes in handy on really busy days. A typical busy day might mean you have to go to your classes, do your laundry, work out, go to a group meeting, and then go out to eat with your friends. It could be tempting to put tasks off based on how you prioritize them (do laundry tomorrow, work out tomorrow, skip class), but chances are it never would have ended up like this if you didn’t put them off to begin with. Every morning when I wake up I just write down a list of the things I have to do that day on a post-it. Whenever I finish something, I check it off. The checking off part is crucial because it gives you a big rush that you just finished a task. That rush motivates me to get more things done and check off the entire list by the end of the day.

Get off social media

Social media is probably the biggest contributing factor to the modern attention span drop. When I need to get something done, I will delete the social media apps from my phone and re-download them when I’m free again. The reason this works is because most of the time, people will be using social media from their phone. Chances are, you won’t be able to re-download the apps back unless you are at home. This makes it so that you can’t cave in and re-download the apps to procrastinate. When you’re not on mobile, you could check out browser extensions that block certain sites like StayFocusd, but the problem with those is that you could easily get rid of them if you wanted to.

Some combination of these three tricks usually end up working for me. One moment I can’t focus and the next my projects are all completed. It feels good to check off that entire list at the end of the day and then binge watch the entire 2nd season of Rick and Morty at home. Hopefully you’ll get your work done, and then you’ll have the free time you’ve always wanted. Go ahead, you deserve it.

How to start performing a flow art

One of my main hobbies is spending time on some sort of flow art. Most of the time I’ll be spinning poi with some friends but you could also find me contact juggling or hooping from time to time. It’s a way for me to have fun and relieve stress from school. It’s hard for me to put into words what exactly it’s all about but I did find a nice quote –

“Flow Arts is a general term used to describe the intersection of a variety of movement-based disciplines including dance, juggling, fire-spinning, and object manipulation.” — The Flow Arts Institute

Whether you want to start a new hobby or just want to have fun, here’s my guide to getting started in the flow arts.

Picking a prop

There are tons of flow props – Poi, Staves, Fans, Hoops, Balls, Wands, Rope darts, Meteors, Puppy hammers, S-staves, clubs, devil sticks, diabolos, and more! The first step is to pick a prop and stick to it. Most guys in the North American flow scene will start with poi and girls with start with hoops but you should pick a prop based on two things: How cool you think it looks, and how fun you think it looks. Here’s some images:

Poi: .

Fans: .

Hoops: .

Staff: .

If you’re looking for something “flow-ey” then I recommend poi. If you’re going for something smooth I recommend contact juggling. If you don’t mind weight and want to be a badass, try out a staff. If you’re into circus stuff, go for devil sticks or diabolos. If you want something simple to chill then look into wands.

Obtaining a prop

Unless you plan on building a prop from scratch, you’re most likely going to be buying one online. Some good sites are Home of Poi, Flowtoys, Master Ong’s prop shop, and The DrexFactor store. If you’re on facebook you can also look for flow arts or circus groups in your area. There are usually local vendors that will sell you their extra props or custom make you props.

Starting out

Your best friend when starting out is patience. When I first started out, my friends were doing insane tricks and I wanted to get up to speed quick but I kept hitting myself. It was frustrating to say the least, but in a way it’s kind of like coding. You need to failing repeatedly to learn anything, and you have to do it a lot. As long as you practice frequently you will feel more comfortable with your prop, even if you aren’t learning a specific move or aren’t making progress, so be as patient as you can. To learn moves, it is best to find a community of artists nearby but if that isn’t possible then look to youtube for tutorials. Home of poi also offers tutorials for any props that you buy off of their website.

In a month of practice you will make significant progress and have a lot of fun as well. A big part of the enjoyment of flow for me is the sense of acheivement from finally learning that new move or showing my friends new tricks. If you keep it up for longer you will also begin to see the fruits of your labor grow into other parts of life. For me it boosted my work ethic, sense of determination, and self-esteem dramatically. You will feel a much greater sense of freedom and maybe find a new respect for other circus and flow arts performers. Overall, I think that learning a prop can be great for anyone and I hope that you start learning one whenever you can!

File Compression Part 2

Here I continue to talk about methods used to compress files. The two discussed here are the Lempel-Ziv (1977) method and Huffman encoding. Both of these (or variants) are used widely in modern file compressors such as WinRar and 7zip. The file compressor I’m working on uses Huffman encoding in particular.

Lempel-Ziv Method

The Lempel-ziv method takes advantage of the fact that in a file, it is very likely that certain strings of text will be repeated. Suppose we have this sentence:

This is a sentence. Here is another sentence.

Notice that certain strings such as “sentence” repeat in the file. We can take advantage of this by using what is called a distance-length pair. A distance length pair takes the form <d,l>. The d is the number of characters you have to go back and the l is the number of characters to copy. To compress the file, we would start reading it and then once we reach the second “sentence.”, we replace it with a distance-length pair as follows.

This is a sentence. Here is another <25,9>

Let’s say that a character is 1 byte long and a distance-length pair is 2 bytes long. We’ve just decreased the file length from 44 to 37. Not much, but the compression ratio increases dramatically as file size increases (>100 bytes). To decompress, we would start writing until we’ve reached the pair, then we copy over what we wrote 25 bytes ago for 9 bytes. This restores the file to the original state.

Although the idea is simple, my example is oversimplified and it turns out that this method can get complicated fast. How do we remember strings to create pairs for later? How do we represent our distance-length pair? How does the decompressor know when it has hit a distance-length pair? Nonetheless, this method is very useful when you have it all figured out.

Huffman Encoding

Huffman encoding takes advantage of the fact that certain characters or strings appear more often than others, and it allows you to represent the frequent characters with fewer bits than the rare characters. The process of Huffman encoding involves creating a Huffman tree which is daunting at first, but extremely useful. Suppose we have this sentence:

She seems silly

The first step to compression is to build a table mapping each character with the number of times it appears. This can be done in O(n) time.

Now that we have this table, we can generate a huffman tree by doing the following (O(nlogn)):

1. Create leaf nodes for each character-frequency pair and insert them into a priority queue (least frequency”
2. Pop 2 elements from the queue, create a new node and set its children to the two popped elements.
3. Set the frequency of the new node to the sum of the frequencies of its children and then insert it into the queue.
4. If there is more than one element in the queue, go back to step 2.
5. You are done, the last element in the queue is the root of the tree.

We should now have a binary tree containing each pair as a leaf node. Using this tree, we can obtain the encodings by the path from the root to each leaf. A left represents ‘0 and a right represents ‘1’. Here are the encodings for this example.

Finally, we can compress the sentence with this mapping to the following:

110000011011101010010111011100111001000001

Note that in the original sentence, each character would have been represented by one byte for a total of 15 bytes or 120 bits. The compressed file was written in only 42 bits! If the decompressor does not know the tree, you would also have to encode the tree into the file. In this example it could be done in 80 bits, bumping the file size to 122 bits. However, the tree size will stay small even if the file size increases, meaning this method is more effective as file size increases.

To decompress, we simply read bit by bit and traverse the huffman tree. If we read a 0 we go left and if we read a 1 we go right. When we hit a leaf node, we just write down the character in that leaf node.

To conclude, the lempel-ziv method is most useful when certain strings repeat in the file. The huffman method works best when certain characters appear very frequently. Both methods work well for large files, and both are widely used in file compressors.

File Compression Part 1

I have recently started working on my own file compressor titled ‘himanZip’. While working, I’ve had to do a lot of research including reading articles, watching videos, and studying protocols. File compression can be really interesting so I’ve decided to write some blog posts detailing what I’ve learned.

Firstly, there are two types of compression:

Lossy Compression: Takes file A and compresses it into a smaller file B. However, decompressing B back to A will result in a loss of quality. This is because some quality is removed in order to compress A to B.

Lossless Compression: Takes file A and compresses it into smaller file B. Decompression from B back to A does not result in any quality loss. The compression only removes redundant information.

To illustrate the quality loss of lossy compression, here is an example: Suppose you are trying to compress an image file of someone’s face. We know that most faces are generally symmetric, so to compress we simply save one half of the person’s face, and to decompress we copy that half over to fully recreate the face again. However, you will lose a lot of the fine detail of the face this way because faces are not actually symmetric.

I will be focusing only on lossless compression because it is inherently more valuable. One simple form of lossless compression is run-length encoding.

Run-length Encoding

Run-length Encoding is a very simple form of compression that makes a lot of assumptions about the format of a file, but is nonetheless useful for illustrating the concept of compression. Suppose we have a simple text file:

Hiiiiiiiiiiiiiiiiiii Weeeeeeeeeeeeeeeeeeoooooooooooooooooooooowwwwwwwwwwwww.

Notice that there are huge stretches of the same type of characters (i,e,o,w). We can take advantage of this and represent these another way. Instead of writing out the huge stretch, we can replace the i’s by (# of i’s)(i). We can do this for each character stretch to compress. The compressed text file would look like this:

H19i W18e22o13w.

After compression, the total character length dropped from 76 to 16! To decompress, everytime we run into a number we just have to copy the following character that many times. Note that this may not work if you actually have numbers in your text file, or if there aren’t a lot of huge streches in the text, although there are ways to get around this.

That’s all for now, in the next post I will talk about Lempel-Ziv(1977) encoding, and Huffman encoding.

New website and Javascript

After another long and hard semester, I have found myself with nothing to do. So, I decided to teach myself some more javascript to revamp my website into one with more personality. I wanted to create a site which served only to show off my projects, and show them off well. This new site would need color, animations, and pictures, all three of which my current site is lacking.

I decided to use javascript as the tool to make my new website look nice because it gives you more control over HTML elements than jQuery does. Also, javascript is an important skill to have in the world of web development, so it was time I learned it.

After about 5 hours of coming up with ideas, searching through W3Schools, and typing up new files, I created a (somewhat) functional prototype. However, I ended up hitting a roadblock which stopped me from turning my website vision into a reality. Ironically, the problem I encountered could’ve been avoided if I had just used a framework such as jQuery. Nonetheless, I did learn a lot and decided to write a blog post about the animation and styling abilities of javascript.

Styling

Anyone who has ever made a website knows that all HTML elements have ‘style’ attributes that can be modified through CSS. These style elements include all sorts of things from height and width to padding and borders to position coordinates. Through javascript, you can access and modify those style elements to edit the look of an HTML element. Specifically, you can write a function that gets called when an event occurs (such as a button click).

function makeRed(elementID) {
  var object = document.getElementById(elementID);
  var originalColor = object.style.color;
  console.log("Original Color: " + originalColor);
   object.style.color = "red";
  console.log("New Color: Red");
}

This is an example javascript function that can take an HTML element by its ID and turn it to the color red. The object is obtained by the .getElementByID() method. Then, the color field in the style can be set to red. To tie this function to the click of a button, you would set the onClick field of the button to “makeRed(elementID);”. Functions like these can be written to change almost all existing attributes of an element.

Animation

This is what makes your website fancy. An animation can’t be created by simply changing attributes with javascript functions normally. However, you can create animations by using an object’s transformation and transition properties. The transform property can be changed like any other style element, except it takes in a function instead of a value. The function specifies what change will happen to the object.

function switchUp() {
  var object = document.getElementById(order[currIndex]);
  var height = object.clientHeight;
   object.style.transition = "transform 1s linear 0s"
  object.style.transform = "translate(0px," + height + "px)";
   currIndex = (currIndex + 1) % currSize;
   var newObject  = document.getElementById(order[currIndex]);
   object.style.transition = "transform 1s linear 0s"
  object.style.transform = "translate(0px," + height + "px)";
}

This is a snippet from my prototype where I shift an entire section up and replace it with a new one from below. Here, the switchUp function is called whenever the user clicks on the up arrow. If the user is on the home section, it will shift the home up and the skills section will come from below to replace it.

In order to acheive this effect, I first obtain the height of the section. Then, I set the transition property to “transform 1s linear 0s”. This means, when the object has to transform, the transformation will take 1 second and the animation will occur in linear time. There will also be a 0 second delay before the transform starts. Then, I actually change the transform property to “translate(0px, height px)”. This translates the section up by height pixels. Finally, I translate the skills section from below up by that same height.

The transform property can also be used to scale up an object or rotate it along any axis.

What I learned

I learned to take HTML elements and transform them to create animations which add flavor and aesthetic to a website. Any element can be edited, so you could have anything from sliding colors to opening doors if you use the javascript properly.

Additionally, I learned that you can use any website to teach you web development. You can view any website’s source files by simply right clicking and selecting “view page source”. By doing this, you can learn how most websites create their animations by looking into the associated javascript and css files. By doing this, you can rapidly learn new things to add to your websites, and also fix existing mistakes within your own site.

Despite the fact that I will probably be going to be at 4:30AM tonight, and that I don’t have a real finished product, I am glad that I’ve learned some useful javascript skills.

The Hungarian Algorithm

Here I’ll talk about the other side project I worked on and the lesson it taught me. This is one that I thought would be easy but turned out to be hard.

This is an algorithm I learned about last semester in my deterministic models class. It is a quick way to solve an assignment problem, which is a problem where you assign m workers to n jobs in order to minimize a cost or maximize a profit. This is not be confused with the quadratic assignment problem, which is NP-hard.

You can read more about it here: Harvard slides

If you’ve ever used the hungarian algorithm before, you’ll know that it is actually pretty straightforward to do on paper. However, there are some things that are a lot easier to do on paper than they are to do on a computer.

The two steps in particular I am talking about are the “covering” step and the “selection” step.

The Covering Step

In this step we are given an m x n matrix with some random numbers in it, some of which are zeroes. Our goal is to cover this matrix with horizontal or vertical lines such that we cover all zeroes with the minimum number of lines.

This is easy to do on paper, but on a computer, how would we do this? At first I tried to find an easy way to do this, such as drawing a vertical and horizontal line at every zero, and then removing unnecessary lines. It turns out this ruins the next step, so I wasn’t able to do this. After spending a lot of time on thinking how to select which lines to draw, I just decided to draw a line in the direction with the most zeroes. Originally this didn’t work, but I made a few tweaks and somehow it managed to pass the tests I put in place.

The Selection Step

In this step we are given an m x n matrix with some random numbers in it, some of which are zeroes. What we must do here is select certain zeroes from the matrix such that each row and each column has exactly one selected zero. If we have more rows than columns or vice versa, then certain rows or columns will not have a zero.

At first I thought I could just go through the rows and select the 0 in the first column that wasn’t already used. This works in some cases, but not all. In the end, I decided I would have to brute force it (O(N^2) at worst). I ended up looking around and found out a cool way to recursively go through a matrix.

//Call this with empty arrays for selection and filledRows. Also use row = 0.

public static boolean hungarianSelect(
  int[][] pivotedMatrix, int[][] selection, int[] filledRows, int row) {
  
  if (row == pivotedMatrix.length)
    return true; // When we reach the bottom of the matrix, we are done.

       
	for(int i = 0; i < pivotedMatrix[row].length; i++){
	  if(pivotedMatrix[row][i] == 0 && filledRows[i] == 0){
	    selection[row][i] = 1;
			filledRows[i] = 1; // If we run into a zero, select it

			if(hungarianSelect(pivotedMatrix,selection,filledRows,row+1))
			  return true;
			selection[row][i] = 0; // If that zero didn't work, reset and continue to the next zero

			filledRows[i] = 0;
		}
	}
	   
	return false;
}

With this, I was able to brute force select zeroes from my matrix which allowed me to finish up the algorithm successfully.

What I learned

The Hungarian algorithm is fairly useful. Although I don’t see myself using it in the foreseeable future, I can see some ways it might be used. The lessons to learn here are that, once again, you should always create a test suite for your projects. It was extremely helpful for this project, as I was always jumping between steps trying to fix new problems that kept on popping up.

Another lesson is that you should not be afraid to look through the internet for help about certain things. Here I did it to figure out a way to brute force through a matrix for the selection step. Yet another lesson would be that you should not be afraid to make helper methods and helper classes. In this particular problem it helped me break down the steps which were complicated on their own. But by breaking them into smaller steps, it became much more manageable.

Be persistent in finding solutions. I spent alot of time trying to implement methods for the steps, and then realizing there was a special case that I was forgetting about. It was frustrating that I had to keep on rewriting the methods, but every time I rewrote the method I learned more about the problem so in the end it was worth it.

The Minimal Perfect HashMap

So over these past couple of weeks I’ve been working on two side projects that I thought were interesting so that I wouldn’t forget some of the computer science I learned over the last semester. Both of the projects were actually much more challenging than I originally thought they were, and doing them both taught me a few lessons about coding do’s and don’ts. I’ll go over one of them here.

While browsing the internet I came across an article about O(1) Hash map lookups by using a method called Minimal perfect hashing. You can read about it here:

DemoFox’s Blog

In short, this is a method that removes collisions entirely from HashMaps which allows for consistent O(1) lookups. The downside is that it requires more memory.

When I started working on this, I originally thought it was basically just a chained hash map with linear probing, and the “salt” values were basically just the location offsets from inside the map. It turns out this does not really work or make sense, and I ran into some problems when implementing. After I looked into it more, it turned out to be a lot more complicated than I thought it was. Essentially, the salt values are a lot more like nonce values. Instead of hashing a key once to find the hashmap position, you actually have to hash a second time with the salt value to find the hashmap position if there was a collision.

// XORing the hashed key with the salt value creates a decent second hash function

private int saltHash(Object key, int salt) {
  return salt ^ key.hashCode();
}

Another thing about the minimal hash map is that it is not meant to be used in the same way as a regular hash map. In the regular hash map, you can put and remove elements freely as well as overwrite certain elements if you put in two elements with the same key.

The minimal hash map however, is meant to be used as a data structure where you put everything in, calculate the hash function, and then access elements. Once you calculate the hash function you are not able to add in more elements and you cannot remove or overwrite elements either.

Its use is somewhat specific, you need to have a set of data that you know will not change, and you must only want to access from that data, and not want to change it in any way. An example of this would be if you were managing a school database. You could create a map of student IDs to student Names, which would then allow you to instantaneously get a student’s name just from knowing their ID, and not having to manually search through a whole database.

What I learned

Overall I think that the Minimal Hashmap is a useful data structure even though the use case is somewhat specific. The lesson I learned here is that when you try to make something, you sure assure yourself that you really know what you are trying to make. Do this by looking through multiple sources, and maybe even trying specific use examples and see if it works. Also, creating test suites is very useful for these scenarios. I originally implemented the whole thing wrong, but because I created a test suite I was able to test my program quickly, and identify the problems properly when I was refactoring.

First blog post!

I have a blog now! I will try to post here every once in a while about projects that I’m working on or even off topic things that I find interesting.

This blog uses jekyll which is a nice static site generator that runs on ruby. It took me a while to get up and running (I actually had to do it on Ubuntu in my virtual machine because it does not support windows) but I really like the way it came out.

Anyways, as of now i’m working on implementing a Minimal perfect hashmap, and the hungarian algorithm in java. You can check them out on my github but I’ve run into some roadblocks so it might take a while before any of them are actually done.

Thanks for checking out my blog!