Making a very large image collection

I am attempting to build a website to provide a focal point for my grandfather's work. He was a popular British illustrator and official war artist. We would like it a filterable archive containing images of all work currently held by the family, numbering several thousand images, and my ambition would be for it to become a complete archive of all the work we can trace, hopefully tens of thousands of images. It is important that each image has some text associated to it with the title, medium, size, current owner etc. I have started to play around with some ideas but I have no idea of the practical limitations of portfolios and galleries or if I should consider adapting a blog. I have bought the universal filter plugin and when I have a clearer understanding of a feasible approach I will get some paid help to get it up and running. My budget is limited. Can anyone give me some advice please?