In condition number of matrix slides, in page 155 it says that the residual is b_hat - A_hatx_hat, but in page 156 it says residual is b - Ax_hat. which one should i use for the calculation?
The second relation (pg 156) assumes that A and b are exactly represented. (I will add this in a later version, but it does not play a role in the assignment.)
In the assignment, you have to take into account that, whenever computer representation comes into play, we are using a two-decimal system. That should be enough to answer the question.