I think you're going to have to turn it around, and post the grid data via ajaxfileupload.
So in your beforeSubmit function, you're going to have to copy the postdata to the file upload plugin, and the submit that.
Not sure which file uploader you're using. I've used the one from http://valums.com/ajax-upload/. With that you would just do:
uploader.setData(postdata);
uploader.submit();
But then you would need to prevent the grid from submitting your request too… there doesnt seem to be an easy way to do that. But you /could/ return [false, “OK”] from beforeSubmit. The problem with that is that the grid would put “OK” into the error field – so you would need to hide the error field immediately.
I think you could do that with a setTimeout at the end of your beforeSubmit.
so (untested, but probably close):
beforeSubmit : function(postdata,form) {