вторник, 23 марта 2010 г.

Record, record, and record once again.

Record, record and record once again.

Files attached to the article:

Let's talk about the records. About those problems and nuances with which we are regularly facing, about those possibilities which SA posesses in the last versions.

Own SA loader has appeared by some reasons. One of the reasons - is widely spread I/Q format, and with other, strangely enough, rather poor quality of records in this format became one of the reasons of its working out, on the one hand.

Poor quality of records in general is a global problem, we considered it here earlier, so there is no sense to repeat, I wish to mention some questions which are not so obvious.

Several I/Q records from recourse http://www.iw3aut.altervista.org/files/samples/index.php

At loading of these files in SA (look attached files), you will be surprised by extremely low level of the records.

Althought, the records apparently, should show all power and possibilities of SDR technology. Unfortunately the result is absolutely opposite. It is impossible to consider neither samples, nor examples of qualitative work.

Level of the record is so low, that at attempts to increase it, there is rather rough granularity and the resulting signal has worse quality than 8 bit record. I would like to remind that 8 bit records are considered by default as unsuitable for tasks of the signal analysis. Such recordS are more suitable for speech digitalization in narrow enought bar.

Dynamic range of such "signals" is extremely low. Spectrally such records look very plainly and lifelessly, I name them as "dead" records, the background noise level, which caused by rough digitization on level, is very high. Such records represent the extremely doubtful material for tasks of signals identification and analysis, in spite of the fact that it I/Q format should provide the highest quality.

This problem is very characteristic for SDR receivers, we have faced it about 4 years ago, however attempts to pay attention to the very bad quality of material for the analysis and identification, hadn't found special understanding, at least the stream of an illiquid material has not decreased at all.

It is curious enough to observe the bad quality of records today, while quality of SDR receivers is constantly grows, digit capacity ADC raises, and the problem is still exists and not going to disappear..

It is may to seem that the heart the problem is insufficient amplification in paths of processing of signal of SDR receivers. In DSP all is co-ordinated in a single whole unit, nothing is taken from anywhere, and nothing disappears into anywhere. If the main ADC has digit capacity in 16 bits then these 16 bits will be allocated on whole length of digitization bar, and if this bar, for example is 40 Mhz, then it is necessary ~600-620 Hz for one position. It means, that if after such ADC you try to select bar in 10 Khz from 40 Mhz, then there is never be high-grade 16 bits, and will be only approximately 20-40 (position)/levels.

It is rough enough example but it reflexts reality. Althought it is only of one possible problems which is linked with digit capacity ADC, which are used in modern SDR receivers.

This problem can be masked easily enought by various filters, switching to higher digit capacity or to arithmetics with a floating point after ADC, but it does not change a real state of affairs, and forces developers to solve uneasy questions.

There are lot of powerfull SDR recievers with powerfull ADC represented today, they are providing a signal's output in 24 or 32 bit format. It would Seem in such situation, that quality of records should be good by default, however it is far not always so.

Dynamic range of 16 bit records is approximately 96 Db, 24 bit is nearby 145 db, 32 bit is about 193 db. If to consider real-life formats which are widely used for records,then for 32 bit numbers floating-point dynamic range will be 700 db, and for 64 bit 7000-8000 db. These are fantastic values of a dynamic range, to display which in a reality on a spectrum is almost not possible, you simply will not see a difference between 650 and 680 db, otherwise the spectrum will remind small cutted chaotic colour salad, which will be hardly possible to understand. qualitative mapping of a complete dynamic range is a certain problem already on 16 bits, and to play back 140-145 db for 24 bit records, very and very uneasy.
Smoothly we approach to the problem. The big dynamic range though guarantees good quality, but does not guarantee automatic solution of all problems, an ability to use all potential power of equipment remains for the person, however not all understand that, and worst of all, most of people do not wish to understand these things that is a little surprising in general.

Typical, even classical example. There is the record 24 bit, that should provide a dynamic range in 140-145 dB.

It is well visible that random impulse interference, completely brings to nothing all advantages of 24 bit formats, as in this record on the main signal we have only about 5000-7000 (!) level/positions, from approximately 16700000 possible. The most part of the dynamic range is spent on exellent representation of the random, single powerful impulse., to the prejudice of other data. And considering that in this record the common bar is 96 Khz and the signal demanding the analysis occupies 3 KHz, then the chances to receive a picture which we got earlier are very great.

This problem is getting worse, because in various cases for the various reasons, 16 bit records are especcialy demanded, and in this case direct conversion makes a bad turn. Usually sound editors are used for these purposes. The file is loaded as 24 or 32 bit and saved as 16. At such approach, the record before saving is scaled, and any program fairly tries to spend this scaling with efforts to provide maximum identity with the original.
In this case, even if the original provides sufficient quality for the analysis, for instance in the previous example on if to give 100000-300000 positions/level on the main signal, and on an impulse 12000000-16000000 positions, then after scaling there will be 65000bits(positions) on an impulse, and nearby 1200-1500 on the main signal. That is, as a result we see maximum detailing of random noise, and rough detailing of nessecary content.

SA preview mode of waveform in the loader, gives possibility to scale a signal before loading, that for the last example guarantees effective allocation of all dynamic range on the main signal, by simple cutting off of everything that exceeds a threshold, as a rule it can be various artefacts or powerful impulse interferences.

This SA possibility is entered for 24 and 32 bit records formats, including the format with a floating point, as 8 and 16 bit data are usually already brought to the necessary scale in advance, and if level of such records(24 and 32 bit) is very small initially, then scaling and amplification will not change situation to the best.

16 bits in overwhelming majority of cases is more than the sufficient value of digitization on level. If to follow a simple rule, then the signal should use as much as possible completely all dynamic range. Complete maximum does not mean from a maximum to a minimum, quite reasonable value is 70 %-80 % of a complete range, the small store always needs to be left, it concerns not only to 16 bit records, but any other format.

Usually the problems with 16 bit digitizations appear, when bars in 10-100 Mhz are inserted in record , in hope after to cut the necessary segments in tens Khz. It is an error. As to SA, SA iscertainly universal analyzer, and allows to make a lot with record, but it is not off-linereceiver, and to assign to it these functions at least is not recommended. SA will perfectly work out with 50 Mhz and above if the signal has bar at ~ 10-20 Mhz, but we cannot guarantee anything if someone intends to cut and analyze signals at 5-10 or 100 Hhz from 50 Mhz. All that should be done at the stage of reception of a signal, that is in RX. Basically, this is is one of the primary goals of the receiver, which for some reason frequently mistakenly supposed to be done by analyzers.

Actually these problems are not such big secret. The huge percent of complete not liquid records walks in the net from one site to another, I with some surprise detected "new" resources where represented "examples" of the various signals, and these "samples" we saw 5-6 years ago, on complenetly another resources.

Good luck!