Recent Posts

6 min read distributed systems

Distributed consensus - i.e., the problem of getting a set of processes to agree on a certain action or value over a network - is a fundamental problem of distributed computing. It arises in settings that range from group membership (i.e., tracking the set of servers in a server group that are currently online) to distributed database commits. And yet, despite its pervasiveness, it is anything but simple to solve.

CONTINUE READING

10 min read programming, Ruby

I have been doing some Ruby programming as part of my recent work ventures, and I think a nice way to learn a new language is trying to understand how to map your previous way of doing things into this new environment, and discovering new things along the way. Having been programming in Python for the best part of the last 4 years, I got used to as a convenient way of structuring certain computations.

CONTINUE READING

17 min read statistics, basics, R

Histograms are probably one the most intuitive ways of performing density estimation in numerical data. In this writeup, I discuss some of their deeper meanings and properties, as I try to digest Wayne Oldford’s excellent lecture on the subject. In particular, we look at how well one can expect to approximate an underlying density as one has access to more samples and utilizes narrower bins. We then discuss a form of histogram that is much less sensitive to the location of bin boundaries, called an Average Shifted Histogram [1].

CONTINUE READING

18 min read algorithms, machine learning, python

Backpropagation is a method for computing gradients of complex (as in complicated) composite functions. It is one of those things that is quite simple once you have figured out how it works (the other being Paxos). In this document, I will try to explain it in a way that avoids the issues I had with most of the explanations I found in textbooks and on the Internet. Inherently, this means that this document is well-suited to my way of thinking, but my hope is that it may also suit yours or, at the very least, may give you a different perspective as you try to grasp how backpropagation works.

CONTINUE READING

Tags

Contact