0.3 Testing

Testing

In order to make direct comparison between two algorithm and two detecting methods that we have discussed in the previous module, we conducted several tests. During each test, we used a 36in*30in board and drew grids every 6 inches. At each intersection point of grids, we generated a sound pulse. On the purpose of producing ideal pulse waves, we knocked two small brass blocks. Six trails were conducted at each point. We record both sound sources’ coordinates and the ones our system measured. After computing the root-mean-square error of six trails at each point, we plot error maps for both x and y coordinates of the testing board.

Testing results

Method I, Direct Equation Solving Method (Threshold Detecting)

As shown in the error map above, this method returns coordinates with large errors. It has root-square-mean error 0.110m for x coordinates and 0.133m for y coordinates. Another problem for this method is that the results from six trails at one point have large variance. The performance of systems is unstable when using Direct Equation Solving Method .

The later analysis shows us that the errors mainly came two aspects. First, we used Matlab build in symbolic toolbox to solve the equation groups, which returns four possible solutions. Despite the fact that the true coordinate lies within those four solutions, there seems no good method that always chooses the right one. Second, the direct physics calculations require high accuracy from hardware. Directly plugging the measured data into equation, the errors are introduced and added up from all the hardware. Therefore, Method I, Direct Equation Solving Method could not fulfill our requirement of high resolution.

Method II, Match Filter (Threshold Detecting)

As shown in the error map above, this method returns coordinates with small errors. It has root-square-mean error 0.035m for x coordinates and 0.043m for y coordinates. Considering the prefixed resolution of our system is 0.01m, our system’s provides good accuracy under Method II, Match Filter (Threshold Detecting).

In this algorithm, we avoided problems that causes large errors in method I. The calculations are numerical and only one result will be returned. The comparison algorithm also avoid calculating result by direct measured data and therefore control the errors coming from hardware. The improvement in performance matches our expectation.

Method II, Match Filter (Peak Detecting)

As shown in the error map above, this method returns coordinates with large errors. It has root-square-mean error 0.138m for x coordinates and 0.136m for y coordinates. Different form Method I, the 6 trails at one point returned relative consistent result. Because of the large errors it returned, this method can not fulfill our resolusion requirements.

Even though using the same match filter algorithm, peak detecting method seems to have a fixed error. Our later analysis attributes this error to the resonance and wave form distortion during transimitting in the air.