I didn’t create the most flexible method of doing a portfolio in my themes, but what is happening is that with different sized thumbnails, it affects the rows below. So if an image is taller, it pushes the container down which then causes the next row to go as you see in your page. The only solution for this is to make sure your featured images for each portfolio item are the same size. It also helps for consistent layout and style too.
To create a layout that has different heights and sometimes even widths, the theme needs a masonry type script that realigns everything and floats them below each other perfectly.