I would line it.
If you line it there will be less stress on the seams of your lovely silk taffeta. The stress would show in pulling around the seams and would eventually tear the fabric.
It will also slide over your body better and be more comfortable to wear.
I have made a silk taffeta shirt/jacket before and didn't line it and regretted it since I was only able to wear it a few times before it looked *worn*
* Advertising and soliciting is strictly prohibited on PatternReview.com. If you find a post which is not in agreement with our Terms and Conditions, please click on the Report Post button to report it.