SA Update: version 6.1.2.0

SA Update version 6.1.2.0

About one or two half months ago, at studying of possibilities of WaveLet conversion, an idea of essential lowering of the common noise level on records has appeared, and recenlty we got an ability to realise this idea in the form of working algorithm.

Conditionally we have named it HRes (HighResolution). Here is comparative screenshots.





Ofcourse, this is relative increasing of resolution quality, but the task of decreasing of the common noise level and selection of spectral lines is rather effectively solved. We see perspective enough application of this algorithm at processing of speech waveforms, signals FSK/MFSK and in automation parameters definition.

We cannot recommend to use this function always, because high detailing of spectra, frequently discourages by an abundance of details more likely, than really helps. But the given method is irreplaceable at the tasks of search of lines of clock frequencies and/or studying of harmonics of a signal, purely these are part of the primary goals of this algorithm and one of the reasons of its development.

The algorithm demands enought resourses, it is recommended to disable HRes at navigation on file , though it is not always notable, but sometimes it can become slow enought. For the same reason, by default this function is switched off.

Good luck!

Record, record, and record once again.

Record, record and record once again.

Files attached to the article:

file1

file2

Let's talk about the records. About those problems and nuances with which we are regularly facing, about those possibilities which SA posesses in the last versions.

Own SA loader has appeared by some reasons. One of the reasons - is widely spread I/Q format, and with other, strangely enough, rather poor quality of records in this format became one of the reasons of its working out, on the one hand.

Poor quality of records in general is a global problem, we considered it here earlier, so there is no sense to repeat, I wish to mention some questions which are not so obvious.

Several I/Q records from recourse http://www.iw3aut.altervista.org/files/samples/index.php

At loading of these files in SA (look attached files), you will be surprised by extremely low level of the records.

Althought, the records apparently, should show all power and possibilities of SDR technology. Unfortunately the result is absolutely opposite. It is impossible to consider neither samples, nor examples of qualitative work.

Level of the record is so low, that at attempts to increase it, there is rather rough granularity and the resulting signal has worse quality than 8 bit record. I would like to remind that 8 bit records are considered by default as unsuitable for tasks of the signal analysis. Such recordS are more suitable for speech digitalization in narrow enought bar.

Dynamic range of such "signals" is extremely low. Spectrally such records look very plainly and lifelessly, I name them as "dead" records, the background noise level, which caused by rough digitization on level, is very high. Such records represent the extremely doubtful material for tasks of signals identification and analysis, in spite of the fact that it I/Q format should provide the highest quality.

This problem is very characteristic for SDR receivers, we have faced it about 4 years ago, however attempts to pay attention to the very bad quality of material for the analysis and identification, hadn't found special understanding, at least the stream of an illiquid material has not decreased at all.

It is curious enough to observe the bad quality of records today, while quality of SDR receivers is constantly grows, digit capacity ADC raises, and the problem is still exists and not going to disappear..

It is may to seem that the heart the problem is insufficient amplification in paths of processing of signal of SDR receivers. In DSP all is co-ordinated in a single whole unit, nothing is taken from anywhere, and nothing disappears into anywhere. If the main ADC has digit capacity in 16 bits then these 16 bits will be allocated on whole length of digitization bar, and if this bar, for example is 40 Mhz, then it is necessary ~600-620 Hz for one position. It means, that if after such ADC you try to select bar in 10 Khz from 40 Mhz, then there is never be high-grade 16 bits, and will be only approximately 20-40 (position)/levels.

It is rough enough example but it reflexts reality. Althought it is only of one possible problems which is linked with digit capacity ADC, which are used in modern SDR receivers.

This problem can be masked easily enought by various filters, switching to higher digit capacity or to arithmetics with a floating point after ADC, but it does not change a real state of affairs, and forces developers to solve uneasy questions.

There are lot of powerfull SDR recievers with powerfull ADC represented today, they are providing a signal's output in 24 or 32 bit format. It would Seem in such situation, that quality of records should be good by default, however it is far not always so.

Dynamic range of 16 bit records is approximately 96 Db, 24 bit is nearby 145 db, 32 bit is about 193 db. If to consider real-life formats which are widely used for records,then for 32 bit numbers floating-point dynamic range will be 700 db, and for 64 bit 7000-8000 db. These are fantastic values of a dynamic range, to display which in a reality on a spectrum is almost not possible, you simply will not see a difference between 650 and 680 db, otherwise the spectrum will remind small cutted chaotic colour salad, which will be hardly possible to understand. qualitative mapping of a complete dynamic range is a certain problem already on 16 bits, and to play back 140-145 db for 24 bit records, very and very uneasy.

Smoothly we approach to the problem. The big dynamic range though guarantees good quality, but does not guarantee automatic solution of all problems, an ability to use all potential power of equipment remains for the person, however not all understand that, and worst of all, most of people do not wish to understand these things that is a little surprising in general.

Typical, even classical example. There is the record 24 bit, that should provide a dynamic range in 140-145 dB.

It is well visible that random impulse interference, completely brings to nothing all advantages of 24 bit formats, as in this record on the main signal we have only about 5000-7000 (!) level/positions, from approximately 16700000 possible. The most part of the dynamic range is spent on exellent representation of the random, single powerful impulse., to the prejudice of other data. And considering that in this record the common bar is 96 Khz and the signal demanding the analysis occupies 3 KHz, then the chances to receive a picture which we got earlier are very great.

This problem is getting worse, because in various cases for the various reasons, 16 bit records are especcialy demanded, and in this case direct conversion makes a bad turn. Usually sound editors are used for these purposes. The file is loaded as 24 or 32 bit and saved as 16. At such approach, the record before saving is scaled, and any program fairly tries to spend this scaling with efforts to provide maximum identity with the original.

In this case, even if the original provides sufficient quality for the analysis, for instance in the previous example on if to give 100000-300000 positions/level on the main signal, and on an impulse 12000000-16000000 positions, then after scaling there will be 65000bits(positions) on an impulse, and nearby 1200-1500 on the main signal. That is, as a result we see maximum detailing of random noise, and rough detailing of nessecary content.

SA preview mode of waveform in the loader, gives possibility to scale a signal before loading, that for the last example guarantees effective allocation of all dynamic range on the main signal, by simple cutting off of everything that exceeds a threshold, as a rule it can be various artefacts or powerful impulse interferences.

This SA possibility is entered for 24 and 32 bit records formats, including the format with a floating point, as 8 and 16 bit data are usually already brought to the necessary scale in advance, and if level of such records(24 and 32 bit) is very small initially, then scaling and amplification will not change situation to the best.

16 bits in overwhelming majority of cases is more than the sufficient value of digitization on level. If to follow a simple rule, then the signal should use as much as possible completely all dynamic range. Complete maximum does not mean from a maximum to a minimum, quite reasonable value is 70 %-80 % of a complete range, the small store always needs to be left, it concerns not only to 16 bit records, but any other format.

Usually the problems with 16 bit digitizations appear, when bars in 10-100 Mhz are inserted in record , in hope after to cut the necessary segments in tens Khz. It is an error. As to SA, SA iscertainly universal analyzer, and allows to make a lot with record, but it is not off-linereceiver, and to assign to it these functions at least is not recommended. SA will perfectly work out with 50 Mhz and above if the signal has bar at ~ 10-20 Mhz, but we cannot guarantee anything if someone intends to cut and analyze signals at 5-10 or 100 Hhz from 50 Mhz. All that should be done at the stage of reception of a signal, that is in RX. Basically, this is is one of the primary goals of the receiver, which for some reason frequently mistakenly supposed to be done by analyzers.

Actually these problems are not such big secret. The huge percent of complete not liquid records walks in the net from one site to another, I with some surprise detected "new" resources where represented "examples" of the various signals, and these "samples" we saw 5-6 years ago, on complenetly another resources.

Good luck!

SA update to version v 6.1.1.7

SA update to version v 6.1.1.7

At prepearing description of signal ROSMODEM two problems were detected.

The first one is - a maximum fragment of a signal, on which the histogram is constructing, is too short for qualitative visualisation.

The second is - the method of construction of the histogram is not enought successful.

The maximum size of a fragment is limited by the value 30-50 kilobyte, these are echoes of the very old not optimised solutions in SA, current SA implementation allows to lift easily enough this boundary to 1-2Mb values, but unfortunly this moment was missed. In this version this error is corrected.

The second problem can't be solved 100 % ideally, but nevertheless, rationing of the whole histogram, allows to receive much more clear better picture of allocation of frequencies and levels, as all lines are visible always completely.

The difference in images is obvious enough and does not demand any remarks.

Actually this is not planned express update. We have in our plans automatic or half-automatic detection both frequency spacing, and freqeuncy quantities, but searches of universal reliable solution can take it's not short time.

Good Luck~

MFSK: Rosmodem

Rosmodem

Amateur mode, Br~15.5 or 1, Sh~15.5, Freqs~144 or more

Here is the sample of this signal, recieved from the real air. Send by Zmej.

Here is an example of work of the new ROS protocol.

(Author): SergUA6

Band Width ~2250-2300 Hz

Low Range ~400 Hz, shift is possible

Baud Rate ~15.5, and very slow mode 1 Hz

Count of Carriers not less than 144

Step between carriers ~15.5 Hz

Pilot tone(s) Start tone ~1481 Hz, with duration ~ 4200 ms,shift is possible

ACF ~18662 ms

RX mode SSB

Sonograms:

pic.1 General view

pic.2 Manipulation speed

Diagrams:

pic.3 Allocation of frequencies and their quantity

pic.4 the signal's ACF and view in VMW

ROSMODEM - the new radio amateur protocol created by Spanish developer Alberto Nieto Ros, the technology of the extension of a spectrum is put in a basis. In this signal if to trust the description, base MFSK-16 extends by the pseudo-casual law at least to MFSK-144. High enough characteristics of this protocol are declared. ACF most likely connected with the period of pseudo-casual sequences of randomization, but this is only supposition. The real transfer speed is essentially lower then speed of manipulation, that can indirectly speak not only about the spectrum extension, but also about usage of serious noise-resistant coding. The mode with speed of manipulation 1 Hz is supported, here are used formerstep between frequencies and principle of creation former.

In addition to the previous update version 6.1.1.6

To be honost we didn't expect such fast reaction on version 6.1.1.5, but we already recieved the letter with request to include support of 32 bit with floating point.

As it was explained to us, a lot of the modern equipment works in this mode, both record and files in this format are not such rarity(as it was possible to think), except that, this format has standard Wave tag = 3, and generally is extended enough.

We solved this task as soon as it was possible.

We would like to represent version 6.1.1.6., where support of 32 bit with floating point is provided.

Since the changes are very local, we have decided do not announce this update as separate one.

Signals Analyzer: update.

SA Update to version v 6.1.1.5

We complete the main changes linked with loading of files of various formats in SA.

Two new elements are inserted:

1 - the area of graphic's construction of a loaded file.

2 - the Switcher with which help the choice of how to interpret the loaded file is realized.

The graphic is constructing for the whole file, that is, the large-scale image by time is output, without possibilities of detailing.

The main objective of mapping of this graphic, is the preliminary scaling of signal's amplitude, tentative estimation of the signal and etc. In many cases it is the extremely useful function. However it demands time, and although speed of construction of the graphic is high enough, about 30-40 mbytes per second, on the big files (20-30) mbytes 5-10 second of time can be demanded, it is linked with features of Windows platform. But basically, all works quickly enough. If the file size exceeds ~157 mbytes (the limit of the loaded data in SA) the graphic is constructing only for this value.

Instead of doubtful function Hot track, much more useful function Auto preview is entered. At it’s activation, the graphic is constructed for any selected file automatically (in case if it possible). Files with zero length, or size shorter then 4 kilobyte are not considered as the data source by SA.

The switcher "Use options" is always switched off by default, and can be activated only for the concrete selected file/record. In this case, the graphic is constructed, pre-scaler becomes accessible, and at loading, the record will be interpreted according to those parameters, which are installed in options.

Such mechanism provides support of 24 and 32 bit records, support of multi-channel wav files, loading of any data within the limits of those formats, which supports SA. In the current version 8, 16, 24 and 32 bit integer data are supported.

24 and 32 bit data have some bytes-representations. SA supports two most wide-spread such as: little-endian used by default and designated as 24A-32A, and big-endian designated as 24B and 32B. Indirectly, the correct choice can be defined by the record graphic, it certainly does not give guarantees in separate special cases, and exhaustive search of variants can be demanded, but such happens seldom enough.

Actually speaking about integer representation, it is necessary to have in view, that numbers with the so-called fixed point are also included into this category and are supported by SA in full volume.

The records in a format with floating point and 64 bit formats, SA does not support. It is not basic limitation, and we do not enter support of these types, only because of big doubts in such necessity. If for someone it is essentially important and necessary, please contact us msm_team[at]live.ru with letter subject 64bit support, and I think the question will easily solved.

Some features and nuances.

The graphic is always is constructed from the supposition that the file is considered as the data, and it is not dependent on a real format. It means, if the file mp3 or wav or what another the graphic it is not the signal/record, which can have a compression, but it is the data in that format, which is exposed/selected in the field "Bits". This data coincides with contents of wav files with Wave tag = 1. This data can also coincide for wav files with other tags, because many programs put their own tags, and write not compressed data, but all this is not guaranteed and does not correspond to contents of mp3 files and to various sound codecs.

Usually multi-channel, not compressed wav files have Wave Tag, which is not equal 1, that locks possibility of their playing with standard resources, for SA however it does not represent a problem as the current loader allows to select any of 16 channels, from any multi-channel standard wav records.

The field "Data start" in sets offset from the file beginning, to the data which will be read by the program at loading. For some, incorrectly issued I/Q records it allows to be placed exactly on the beginning of countings, we have met a lot of such records unfortunately.

Pre-scaler is accessible only in a mode of activated function "Use options", it sets the common coefficient of scaling at file loading. Pre-scaler allows to lift signal level on weak records or, that is more important, to neglect strong interferences, by having lifted an average level of record to the normal. Typical enough problem at conversion of 24 or 32 bit records. Pre-scaler is dumped in an initial state, every time at change of the channel, data format, or the record.

Good Luck~