<< Chapter < Page Chapter >> Page >
This module covers probability density estimation of signals assuming a knowledge of the first order amplitude distribution of observed signals. It describes Type and Histogram Estimators as well as density verification of the estimates.

Probability density estimation

Many signal processing algorithms, implicitly or explicitly, assume that the signal and the observation noise are each welldescribed as Gaussian random sequences. Virtually all linear estimation and prediction filters minimize the mean-squarederror while not explicitly assuming any form for the amplitude distribution of the signal or noise. In many formal waveformestimation theories where probability density is, for better or worse, specified, the mean-squared error arises fromGaussian assumptions. A similar situation occurs explicitly in detection theory. The matched filter is probably the optimumdetection rule only when the observation noise is Gaussian. When the noise is non-Gaussian, thedetector assumes some other form. Much of what has been presented in this chapter is based implicitly on a Gaussian model for both the signal and the noise. When non-Gaussian distributions areassumed, the quantities upon which optimal linear filtering theory are based, covariance functions, no longer suffice tocharacterize the observations. While the joint amplitude distribution of any zero-mean, stationary Gaussian stochasticprocess is entirely characterized by its covariance function; non-Gaussian processes require more. Optimal linear filteringresults can be applied in non-Gaussian problems, but we should realize that other informative aspects of the process arebeing ignored.

This discussion would seem to be leading to a formulation of optimal filtering in a non-Gaussian setting. Would that suchtheories were easy to use; virtually all of them require knowledge of process characteristics that are difficult tomeasure and the resulting filters are typically nonlinear [Lipster and Shiryayev: Chapter 8] Rather than present preliminary results, we take the tack that knowledge is better than ignorance: At least thefirst-order amplitude distribution of the observed signals should be considered during the signal processing design. Ifthe signal is found to be Gaussian, then linear filtering results can be applied with the knowledge than no otherfiltering strategy will yield better results. If non-Gaussian, the linear filtering can still be used and theengineer must be aware that future systems might yield "better" results. Note that linear filtering optimizes the mean-squared error whether the signalsinvolved are Gaussian or not. Other error criteria might better capture unexpected changes in signal characteristicsand non-Gaussian processes contain internal statistical structure beyond that described by the covariancefunction.

Types

When the observations are discrete-valued or made so by digital-to-analog converters, estimating the probability massfunction is straightforward: Count the relative number of times each value occurs. Let

    r 0 r L 1
denote a sequence of observations, each of which takes on 𝒜 a 1 a N . This set is known as an alphabet and each a n is a letter in that alphabet. We estimate the probability that an observation equals one of the letters according to P r a n 1 L l 0 L 1 I r l a n where I · is the indicator function, equaling one if its argument is true and zero otherwise. This kind of estimate is known ininformation theory as a type [Cover and Thomas: Chapter 12] , and types have remarkable properties. For example, if theobservations are statistically independent, the probability that a given sequence occurs equals r r 0 r L 1 l 0 L 1 P r r l Evaluating the logarithm, we find that r P r r l Converting to a sum over letters reveals
r n 0 N 1 L P r a n P r a n L n 0 N 1 P r a n P r a n P r a n P r a n L P r P r P r
which yields
r L P r P r P r
We introduce the entropy [Cover and Thomas: §2.1] and Kullback-Leibler distance [See Stein's Lemma ]. P n 0 N 1 P a n P a n P 1 P 0 n 0 N 1 P 1 a n P 1 a n P 0 a n Because the Kullback-Leibler distance is non-negative, equaling zero only when the two probability distributions equal each other, we maximize [link] with respect to P by choosing P P : The type estimator is the maximum likelihood estimator of P r .

Questions & Answers

what does the ideal gas law states
Joy Reply
Three charges q_{1}=+3\mu C, q_{2}=+6\mu C and q_{3}=+8\mu C are located at (2,0)m (0,0)m and (0,3) coordinates respectively. Find the magnitude and direction acted upon q_{2} by the two other charges.Draw the correct graphical illustration of the problem above showing the direction of all forces.
Kate Reply
To solve this problem, we need to first find the net force acting on charge q_{2}. The magnitude of the force exerted by q_{1} on q_{2} is given by F=\frac{kq_{1}q_{2}}{r^{2}} where k is the Coulomb constant, q_{1} and q_{2} are the charges of the particles, and r is the distance between them.
Muhammed
What is the direction and net electric force on q_{1}= 5µC located at (0,4)r due to charges q_{2}=7mu located at (0,0)m and q_{3}=3\mu C located at (4,0)m?
Kate Reply
what is the change in momentum of a body?
Eunice Reply
what is a capacitor?
Raymond Reply
Capacitor is a separation of opposite charges using an insulator of very small dimension between them. Capacitor is used for allowing an AC (alternating current) to pass while a DC (direct current) is blocked.
Gautam
A motor travelling at 72km/m on sighting a stop sign applying the breaks such that under constant deaccelerate in the meters of 50 metres what is the magnitude of the accelerate
Maria Reply
please solve
Sharon
8m/s²
Aishat
What is Thermodynamics
Muordit
velocity can be 72 km/h in question. 72 km/h=20 m/s, v^2=2.a.x , 20^2=2.a.50, a=4 m/s^2.
Mehmet
A boat travels due east at a speed of 40meter per seconds across a river flowing due south at 30meter per seconds. what is the resultant speed of the boat
Saheed Reply
50 m/s due south east
Someone
which has a higher temperature, 1cup of boiling water or 1teapot of boiling water which can transfer more heat 1cup of boiling water or 1 teapot of boiling water explain your . answer
Ramon Reply
I believe temperature being an intensive property does not change for any amount of boiling water whereas heat being an extensive property changes with amount/size of the system.
Someone
Scratch that
Someone
temperature for any amount of water to boil at ntp is 100⁰C (it is a state function and and intensive property) and it depends both will give same amount of heat because the surface available for heat transfer is greater in case of the kettle as well as the heat stored in it but if you talk.....
Someone
about the amount of heat stored in the system then in that case since the mass of water in the kettle is greater so more energy is required to raise the temperature b/c more molecules of water are present in the kettle
Someone
definitely of physics
Haryormhidey Reply
how many start and codon
Esrael Reply
what is field
Felix Reply
physics, biology and chemistry this is my Field
ALIYU
field is a region of space under the influence of some physical properties
Collete
what is ogarnic chemistry
WISDOM Reply
determine the slope giving that 3y+ 2x-14=0
WISDOM
Another formula for Acceleration
Belty Reply
a=v/t. a=f/m a
IHUMA
innocent
Adah
pratica A on solution of hydro chloric acid,B is a solution containing 0.5000 mole ofsodium chlorid per dm³,put A in the burret and titrate 20.00 or 25.00cm³ portion of B using melting orange as the indicator. record the deside of your burret tabulate the burret reading and calculate the average volume of acid used?
Nassze Reply
how do lnternal energy measures
Esrael
Two bodies attract each other electrically. Do they both have to be charged? Answer the same question if the bodies repel one another.
JALLAH Reply
No. According to Isac Newtons law. this two bodies maybe you and the wall beside you. Attracting depends on the mass och each body and distance between them.
Dlovan
Are you really asking if two bodies have to be charged to be influenced by Coulombs Law?
Robert
like charges repel while unlike charges atttact
Raymond
Got questions? Join the online conversation and get instant answers!
Jobilize.com Reply

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now




Source:  OpenStax, Statistical signal processing. OpenStax CNX. Dec 05, 2011 Download for free at http://cnx.org/content/col11382/1.1
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Statistical signal processing' conversation and receive update notifications?

Ask