44 inch pounds is more than finger tight. 30-50 inch pounds is about the max torque the average person can twist on a screwdriver handle comfortably when using one hand. That's why most torque devices that go over 50 inch pounds switch to at least a pistol grip or T-handle design to give the operator a grip that allows for more comfortable force transmission during use.
I did some quick looking around online. I am assuming we're talking about a good quality alloy socket head set screw threaded into steel, not threaded into aluminum. Here is the best chart I found after doing a few quick online searches. It recommends 4Nm (or 35inlbs) seating torque for a M5 socket head set screw with a cup style or a knurled cup style point.
Look at the right column of the chart near the bottom of the page.
Another source, but this one is in SAE only. M5 is only slightly larger in diameter than #10, so add a small amount to the torque recommendation for #10 set screw and you should be about right. Their chart shows 33inlbs recommended for a #10 setscrew, so the 35inlb recommendation from the first link seems to now have some secondary verification.
AFT Fasteners provides a helpful resource on socket set screw sizes / dimensions, material, heat treatment, hardness and torsional strength.
www.aftfasteners.com
It doesnt sound like the 44inlbs the OP was using is too far off, but it looks like he should ideally reduce his torque down about 20-25%.
I also included a screenshot of the tables and attached them to this post in case the links stop working in the future.