Skip to content
Snippets Groups Projects
Commit c3d85fd0 authored by Daniel Maier's avatar Daniel Maier
Browse files

cholesky: fix calc_error to prevent nan

parent eda5b71f
No related branches found
No related tags found
No related merge requests found
......@@ -100,7 +100,7 @@ void calc_error(int n,
e = fabs(accurate[i][j] - test[i][j]);
}
/* clamp error to max 1 */
if(e > 1)
if(e > 1 || e != e)
e = 1;
err += e;
x++;
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment