Sounds like Matrix is the right field for your situation. The harder part is how you decide to implement this on the front-end.
While your question doesn't say it explicitly, it is tagged plugin-sproutforms
so I'm going to base my answer on how to approach this using Sprout Forms.
Option 1: Custom HTML
If you only have a single form, building things out in Custom HTML might be worth considering.
You can grab the SproutForms_FormModel
using the form
tag:
{% set form = craft.sproutForms.form('formHandle') %}
Or use the displayTab and displayField tags to output parts of your form that you don't need additional control over.
You would need to build out support for the Matrix Field manually, but Sprout Forms can receive data for a Matrix Field that is built in the proper format.
Option 2: Matrix Custom Field Plugin
If you need to reuse your form in multiple places, or have the need to reuse Matrix fields on the front-end, you might consider building a Custom Field for Sprout Forms that adds front-end Matrix support.
There are several examples in the Sprout Fields plugin of how to add front-end Custom Field support for Sprout Forms, it would just require a bit more work to take into account all the scenarios that a Matrix field could present. Depending on your scenario, you might be able to simplify things and just add support for the fields you need to start off, and add more complexity as the need arose.