Gr-baz

General

gr-baz is a GNU Radio project that adds new functionality (blocks, GRC definitions, apps, etc). It uses the standard GNU Radio block source tree layout and build process.

Part of the new functionality makes it easier to analyse signals. For example, performing cyclostationary analyis (see Variable Delay block below) and Fast Auto-correlation. You can then do things like:

Blind signal analysis of satellite transmissions
- I presented this, along with my Mode S RADAR 3D airspace visualisation, in a talk entitled "Hacking the wireless world with Software Defined Radio" at Ruxcon 2011
OFDM/DRM analysis
- I put together a video on analysing signals on HF with these blocks in GNU Radio

NOTE: A few components will not work (e.g. BorIP client) without applying certain patches to your GNU Radio source. Please see below.

The module name is baz, so in Python one would write:

baz.<block>(args)

To import a file (e.g. borip):

from baz import <file>

If you make any improvements on this code, please get in touch!

Source code

The code is available on github:

https://github.com/balint256/gr-baz

NOTE: You must have libusb-1.0 header+lib available! This is often easily installed using your favourite package manager. (I haven't yet had the time to fix the Automake configure script to check for it.)

The old pre-3.7 code is still on SVN, or can be browsed with HTTP (you probably don't want this):

http://svn.spench.net/main/gr-baz/

Compilation

CMake

Note: gr-baz now uses CMake! For example you can:

cd ~/build/<module name>
cmake ~/src/<module name>

Or even better use cmake-gui.

Autoconf/make (deprecated)

autoconf/make are still supported, but new blocks will only be built using CMake (i.e. the old build system will be frozen):

Please follow these steps:

./bootstrap
./configure
make
sudo make install

NOTE: bootstrap is a shell script, so if it is not already executable, run:

chmod +x bootstrap

or instead:

sh bootstrap

If you encounter complaints about not finding libraries, you may also need a:

sudo ldconfig

Components

patch

Before using various blocks in gr-baz, you must apply at least some of the patches contained in this directory to your GNU Radio source tree. For example: you must update gr_udp_source if you wish to use it as a BorIP client.

msvc

Contains projects enabling compilation under Microsoft Visual Studio on Windows.

librtl2832++

This Visual Studio project for compiling librtl2832++ under Windows, which will produce the DLL enabling use of RTL2832U-based devices on Windows (for example: the ExtIO plugin for HDSDR).

Note: You will need to install the WinUSB (libusb1) driver for your USB device before it will work with librtl2832++.dll.

lib (C++)

librtl2832++

This is the primary file for librtl2832++, which presents a simple C++ interface for (abstracted) control of the RTL2832U demodulator chip and associated tuners for use as a cheap Software Defined Radio receiver.

rtl_source_c

Updates

The RTL2832U-related information has been moved to its own page.

For updates, follow me @spenchdotnet.

I have created a new discussion forum on Google Groups as a place to discuss this code.

For feedback, please get in touch.

Change list

SVN revision/git commit:

r668/d03c816: Completely re-written, new GRC block
r677/47e6641: Support for FC0012/FC2580, compilation of librtl2832++ on Windows (MSVC)
r679/51ba95b: More devices (DIKOM, Dexatek 3)
r687/0feb77e: Auto-probe tuners! Also more devices (Twintech, Genius TVGo, SVEON)

General

This is a straight-forward block for receiving samples directly from a USB DVB-T stick that uses the Realtek RTL2832 demodulator and associate tuners (see list below).

The demodulator itself samples at 8-bit I/Q, and then this block re-processes this to output complex values (gr-complex).

It also supports:

LNA gain control (the gain range depends on the tuner)
for the E4000: optional automatic tuner mode control (enabled via auto_tuner_mode. This is activated when changing the gain and will calculate whether to use the nominal/sensitive/linear tuner mode).

For RTL2832U-specific information, please see the separate page.

If you want to use this on Windows, get the USRP Interfaces plugin and watch the installation instructions and demo.

Videos

There are new demo videos showing it receiving:

an encrypted P25 transmission, which is decrypted and decoded by OP25. (See the screenshot below for the flowgraph and new OP25 Decoder block, which is in gr-baz.)
aviation RADAR signals (Mode S ADS-B), which are decoded by Modez and visualised with Aviation Mapper.

You can watch the original demo video of this block running from GRC.

Notes

Internally the source block is multithreaded to ensure smooth buffering of samples from the device, and delivery to GNU Radio's runtime.

If the USB device is disconnected at runtime, the block detects this and signals EOF to the runtime.

I have modified the operation of the E4000 tuner to yield (in my opinion) better receiver performance. Specifically I have fixed it to manual gain control and also disabled the DC offset loop as on my device it was the cause of annoying interference just to the right of 0 Hz (the LO).

This block requires libusb-1.0 to compile. Please have the development files installed.

The block outputs four short strings to notify you of streaming problems:
- rO: Overflow inside libusb
- rB: Overrun in streaming buffer (buffer consumption is too slow - have you set your resampling rates correctly?)
- rT: The device did not yet submit new samples by the time it should have (i.e. the calculated libusb read interval + a little extra = 'Wait delay'), but the consumer is already asking for more samples - so we immediately give the consumer the samples from the buffer to avoid a pause in processing. Too many of these and...
- rU: The internal buffer has underrun after too many consumer requests and not enough data coming in from the device via libusb. The block will now re-buffer to the 'Buffer level' (per cent) before sending any more samples to the consumer.

Information, and a simple Linux command-line capture utility, for the RTL2832 demodulator can be found at rtl-sdr on Osmocom. Thank you to Antti Palosaari for uncovering this capability of the chip, and Steve Markgraf for putting together rtl-sdr!

Permissions

To run with non-elevated privileges, choose <group name, e.g. 'dvb'>, and as root:

/usr/sbin/groupadd [group name]
/usr/sbin/usermod -G [group name] -a [user name]

Create the file:

/etc/udev/rules.d/10-[pick a name].rules

and add the following to it (for older udev):

# rule to grant read/write access on USRP to group named usrp.
# to use, install this file in /etc/udev/rules.d as 10-usrp.rules
ACTION=="add", BUS=="usb", SYSFS{idVendor}=="[your VID]", SYSFS{idProduct}=="[your PID]", GROUP:="[group name]", MODE:="0660"

Or if you have a modern udev:

ACTION=="add", SUBSYSTEM=="usb", ATTRS{idVendor}=="[your VID]", ATTRS{idProduct}=="[your PID]", GROUP="[group name]", MODE="0666"

Then:

invoke-rc.d udev reload (or restart if you wish)

Have a look to see if the group for your device has changed (and mode should be crw-rw----):

ls -lR /dev/bus/usb

Then restart (unfortunately).

Issues

If, when you run the flowgraph, libgnuradio-baz.so.0 cannot be found, run:

sudo ldconfig

For other issues specific to the RTL2832U hardware, please see the separate page.

Interface

GR_SWIG_BLOCK_MAGIC(baz,rtl_source_c);

baz_rtl_source_c_sptr baz_make_rtl_source_c (bool defer_creation = false);

class baz_rtl_source_c : public gr_sync_block
{
private:
  baz_rtl_source_c(bool defer_creation = false);
public:
	void set_defaults();
	bool create(bool reset_defaults = false);
	void destroy();
public:
	void set_vid(/*uint16_t*/int vid);
	void set_pid(/*uint16_t*/int pid);
	void set_default_timeout(int timeout);	// 0: use default, -1: poll only
	void set_fir_coefficients(const std::vector</*uint8_t*/int>& coeffs);
	void set_crystal_frequency(uint32_t freq);
	void set_tuner_name(const char* name);
public:
	size_t recv_samples_per_packet() const;
	uint64_t samples_received() const;
	uint32_t overflows() const;
	bool running() const;
	uint32_t buffer_size() const;
	uint32_t buffer_times() const;
	bool buffering() const;
	uint32_t read_packet_count() const;
	uint32_t buffer_overflow_count() const;
	uint32_t buffer_underrun_count() const;
public:
	void set_verbose(bool on = true);
	void set_read_length(/*uint32_t*/int length);
	void set_buffer_multiplier(/*uint32_t*/int mul);
	void set_use_buffer(bool use = true);
	void set_buffer_level(float level);
public:
	bool relative_gain() const;
	bool verbose() const;
	uint32_t read_length() const;
	uint32_t buffer_multiplier() const;
	bool use_buffer() const;
	float buffer_level() const;
public:
	bool set_sample_rate(double sample_rate);
	bool set_frequency(double freq);
	bool set_gain(double gain);
	bool set_bandwidth(double bandwidth);
	bool set_gain_mode(int mode);
	bool set_gain_mode(const char* mode);
	void set_relative_gain(bool on = true);
	int set_auto_gain_mode(bool on = true);
public:
	double sample_rate() const;
	RTL2832_NAMESPACE::range_t sample_rate_range() const;
	double frequency() const;
	double gain() const;
	double bandwidth() const;
	int gain_mode() const;
	std::string gain_mode_string() const;
	bool auto_gain_mode() const;
public:	// SWIG get: tuner ranges/values
	RTL2832_NAMESPACE::range_t gain_range() const;
	RTL2832_NAMESPACE::values_t gain_values() const;
	RTL2832_NAMESPACE::range_t frequency_range() const;
	RTL2832_NAMESPACE::range_t bandwidth_range() const;
	RTL2832_NAMESPACE::values_t bandwidth_values() const;
	RTL2832_NAMESPACE::num_name_map_t gain_modes() const;
	std::pair<bool,int> calc_appropriate_gain_mode()/* const*/;
};

Screenshots

rtl_source_c also works with OP25. Please see the OP25 Decoder block below!

Channel selection and FM demodulation with RTL2832 Source

Example of the GRC Source block in a flowgraph

The above flowgraph running

delay

Unlike the original gr_delay block that has a fixed delay, this allows for a variable delay that can be changed at runtime. The delay can be positive or negative. If any padding is required, it uses the value of the last sample.

This block is particularly useful for investigating periodic repetitions within signals (e.g. adjusting cyclostationary lag when performing blind analysis on an OFDM signal).

GR_SWIG_BLOCK_MAGIC(baz,delay)

baz_delay_sptr baz_make_delay (size_t itemsize, int delay);

class baz_delay : public gr_sync_block
{
 private:
  baz_delay (size_t itemsize, int delay);

 public:
  int  delay() const;
  void set_delay (int delay);
};

Using the Variable Delay block to adjust the delay of samples into Complex Conjugate/Multiply blocks in order to find the cyclostationarity of the input signal (i.e. perform cyclostationary analysis to determine the periodicity of the signal to find the symbol or baud rate of the raw modulated data - helpful for blind signal anaylsis).

pow_cc

Raise incoming signal to a power (exponent), and optionally divide resulting signal by 10^div_exp to ensure it doesn't go too high and result in NaNs/Infinity in subsequent floating-point calculations.

GR_SWIG_BLOCK_MAGIC(baz,pow_cc);

baz_pow_cc_sptr
baz_make_pow_cc (float exponent, float div_exp = 0.0);

class baz_pow_cc : public gr_sync_block
{
  baz_pow_cc (float exponent, float div_exp = 0.0);
public:
  void set_exponent(float exponent);
  void set_division_exponent(float div_exp);
  float exponent() const;
  float division_exponent() const;
};

Using the Power block to determine the order of a PSK signal (i.e. determine the n of a phase-shift keyed signal, e.g. BPSK, QPSK, 8PSK).

print_char

Print bytes to the console/stdout (or optionally a file). The hexadecimal values of the bytes are printed, rather than the raw character interpretation of the value.

This block accepts a byte-stream input (containing the data to print), and an optional float-stream (a variable length signal to be used as the trigger for printing). The threshold can be set to only print bytes when the values in the float-stream exceed the specified threshold. To prevent too much data being printed, set limit to the total number of characters to be printed. This is reset each time the float-stream drops below the trigger threshold (i.e. it can be used to print the first limit demodulated bytes when the receiver becomes unsquelched, e.g. VDL Mode 2).

GR_SWIG_BLOCK_MAGIC(baz,print_char);

baz_print_char_sptr baz_make_print_char (float threshold = 0.0, int limit = -1, const char* file = NULL);

class baz_print_char : public gr_sync_block
{
private:
  baz_print_char (float threshold, int limit, const char* file);
};

puncture_bb/depuncture_ff

Puncture a stream of bytes after Forward Error Correction to reduce the code rate from that of the mother code.

Also, depuncture a stream of floats (symbols) to restore the rate of the received code to that of the mother code prior to detecting the code for errors. Missing symbol slots are marked by erasures (the value 0 is inserted into the outgoing float symbol stream).

The matrix parameter can be set at design and runtime. Here are some examples taken from autofec (see below):

_puncture_matrices = [ # Format is ('name', matrix, rate ratio)
  ('1/2', [1,1], (1, 2)),
  ('2/3', [1,1,0,1], (2, 3)),
  ('3/4', [1,1,0,1,1,0], (3, 4)),
  ('5/6', [1,1,0,1,1,0,0,1,1,0], (5, 6)),
  ('7/8', [1,1,0,1,0,1,0,1,1,0,0,1,1,0], (7, 8)),
  ('2/3*', [1,1,1,0], (2, 3)),
  ('3/4*', [1,1,1,0,0,1], (3, 4)),
  ('5/6*', [1,1,1,0,0,1,1,0,0,1], (5, 6)),
  ('7/8*', [1,1,1,0,1,0,1,0,0,1,1,0,0,1], (7, 8))

GR_SWIG_BLOCK_MAGIC(baz,puncture_bb)

baz_puncture_bb_sptr baz_make_puncture_bb (const std::vector<int>& matrix);

class baz_puncture_bb : public gr_block
{
 private:
  baz_puncture_bb (const std::vector<int>& matrix);

 public:
  void set_matrix (const std::vector<int>& matrix);
};

GR_SWIG_BLOCK_MAGIC(baz,depuncture_ff)

baz_depuncture_ff_sptr baz_make_depuncture_ff (const std::vector<int>& matrix);

class baz_depuncture_ff : public gr_block
{
 private:
  baz_depuncture_ff (const std::vector<int>& matrix);

 public:
  void set_matrix (const std::vector<int>& matrix);
};

swap_ff

Swap pairs of incoming float samples (if disabled, acts in pass-through mode).

GR_SWIG_BLOCK_MAGIC(baz,swap_ff)

baz_swap_ff_sptr baz_make_swap_ff (bool bSwap);

class baz_swap_ff : public gr_sync_block
{
 private:
  baz_swap_ff (bool bSwap);

 public:
  void set_swap (bool bSwap);
};

test_counter_cc

Used to check a simulated complex stream. The I and Q values are not a real signal; they are values that should count upward together. If values are dropped, or go out of sync, this block will report problems to stderr.

A source of such a test stream is the ExtIO_USRP plugin. To enable 'test mode', add the following DWORD to the registry:

HKEY_CURRENT_USER\<Winrad/HDSDR/WRplus/etc>\ExtIO_USRP\Settings\m_bTestMode = 1

GR_SWIG_BLOCK_MAGIC(baz,test_counter_cc);

baz_test_counter_cc_sptr baz_make_test_counter_cc ();

class baz_test_counter_cc : public gr_sync_block
{
  baz_test_counter_cc ();
};

unpacked_to_packed_bb

This is the missing combination of operations of those blocks included in the GNU Radio source tree. This one here will take bits_per_chunk from the incoming byte stream and reassemble outgoing bytes to contain bits_into_output bits from the original stream. For example: in VDL Mode 2 the D8PSK demodulator outputs an unpacked byte stream (which is really the equivalent of a bit stream). This block can be used to reassemble the individual 8PSK symbols into individual bytes (i.e. 3 bits in a byte, so output value ranges from 0 to 7).

GR_SWIG_BLOCK_MAGIC(baz,unpacked_to_packed_bb);

baz_unpacked_to_packed_bb_sptr
baz_make_unpacked_to_packed_bb (unsigned int bits_per_chunk, unsigned int bits_into_output, /*gr_endianness_t*/int endianness = GR_MSB_FIRST);

class baz_unpacked_to_packed_bb : public gr_block
{
  baz_unpacked_to_packed_bb (unsigned int bits_per_chunk, unsigned int bits_into_output, /*gr_endianness_t*/int endianness);
};

agc_cc

Experimental and unfinished.

raw2num

Originally from here. Source modified to make offset & length arguments optional.

Converts binary stream of real floats, complex floats, ints and incorrectly-signed/offset ints to readable text, which can then be input to GNU Plot.

Note: these files are not compiled by the build system. To build them manually, simply type gcc <source filename> -o <binary output filename> and place it in a directory on your PATH (e.g. ~/bin).

python

Legacy usrp

Before USRP Hardware Driver (UHD) because the standard way to communicate with all USRPs through a uniform API, the (now) Legacy USRP interface libusrp was used to talk to the USRP 1 & 2 (its source was distributed inside the GNU Radio source tree).

As the legacy libusrp is no longer found in GNU Radio, older applications that stream data to/from USRPs need to be updated to use the new UHD interface. This is a straightforward task, however sometimes you may want an application to 'just work'.

To this end, usrp.py can be placed (symlinked) into the GNU Radio Python installation directory to provide the same module the Legacy USRP interface had:

sudo ln -s <path to...>/gr-baz/python/usrp.py /usr/local/lib/python2.7/dist-packages/gnuradio/usrp.py

Note: Please check your Python version for the above destination path.

Now you can do the following, and it will calls the appropriate UHD functions underneath:

from gnuradio import usrp

# Inside Pythonic GNU Radio block's function:
    self.usrp = usrp.source_c()
    self.usrp.set_decim_rate(decimation_rate)
    if subdev_spec is None:
        subdev_spec = usrp.pick_rx_subdevice(self.usrp)
    self.usrp.set_mux(usrp.determine_rx_mux_value(self.usrp, subdev_spec))
    subdev = usrp.selected_subdev(self.usrp, subdev_spec)
    capture_rate = self.usrp.adc_freq() / self.usrp.decim_rate()
    if gain is None:
        g = subdev.gain_range()
        gain = float(g[0]+g[1])/2
        subdev.set_gain(gain)
    r = self.usrp.tune(0, subdev, frequency)

This does not include the XML block definition for GNU Radio companion, but you will be able to find this in old versions of the GNU Radio source tree (I will look at including it in gr-baz at a later date).

There are two ways of instantiating the underlying uhd_usrp_source:

# (1) usrp.source_c constructor:
def __init__(self, address=None, which=0, decim_rate=0, adc_freq=64e6, defer_creation=True)
#   'address' is UHD device address
#   'which' is currently ignored
#   'decim_rate' is decimation rate
#   'adc_freq' is master clock frequency (default is USRP1 64MHz clock)
#   'defer_creation' when True means underlying uhd_usrp_source will be instantiated when calling selected_subdev (see below)
#                 ...otherwise (when False) it will be instantiated in the constructor

# (2) When calling instance's 'selected_subdev':
def selected_subdev(self, subdev_spec):
# ...or global version:
def selected_subdev(u, subdev_spec):

For more information on which functions are available in the shim, please consult the source.

op25

OP25 decoder block, which encapsulated FSK4 demodulator and OP25 decoder.

Accepts float baseband input, outputs voice and optionally fine-tune messages (into a Message Callback block) and dibits from demodulator to judge how well the receiver is working/how strong the signal is.

You can watch a demo video of it decrypting an encrypted P25 transmission captured using rtl_source_c for RTL2832U-based DVB-T USB devices.

RTL2832 Source feeding new OP25 Decoder block (flowgraph available in samples)

message_callback

Given a message queue, starts a daemon thread that waits on the queue. When a new message arrives, it calls the supplied callback with the first argument of the message. (Warning: this is actually not thread-safe).

This is handy because you can use messages generated by blocks to update variables/GUI elements/etc in a flowgraph (i.e. implement a feedback mechanism), instead of only having one-way GUI control over the blocks.

borip

NOTE: You must apply the appropriate patch to enable BorIP support in gr_udp_source first!

Enables BorIP client support in GNU Radio. Either access this class directly in hand-written Python code, via the GRC block (see below), or enable seamless connection to a remote USRP exposed to your LAN with BorIP server.

BorIP has some neat features for using a USRP 1 over a network (more details in the BorIP specification):

Data packets contain a header (optional, enabled by default) that enables the receiver to keep track of capture overruns on the server, and packets lost on the network.
Transparent operation (see below)
Automatically handle connection interruption without affecting the flowgraph's operation: if contact is lost with the BorIP server (e.g. network is unplugged, server power lost, etc), the UDP data stream would stop, and eventually the server would stop sending data and disconnect after the TCP control connection closes. The BorIP client can be configured to automatically reconnect to the server in the background and start streaming data again as if nothing had happened.

The UDP receiving code will process BorIP packets and output losses (bO) and incorrect payload sizes (b!) to stderr. uO is also output when the remote server experiences a capture overrun.

Operational modes

Traditional UDP Source that handles BorIP data packets
- NOTE: There is no TCP control channel established (unlike the proper BorIP Source block), so control commands cannot be sent to a server. This mode is suited for use with the UDP Relay option in the ExtIO_USRP+FUNcube Dongle plugin.

The original UDP Source block with added BorIP support (also features the Test Counter block)

Proper BorIP Source that also establishes a control channel to communicate radio configuration to the server
- This block operates more like a traditional USRP Source block in that one can configure parameters such as sample rate, frequency, gain and antenna:

The new BorIP Source block with additional configurable parameters

To enable seamless support, add the following to gr-usrp/src/__init__.py:

import sys
if not sys.modules.has_key('baz.borip'):
    from baz import borip

This tells Python to import borip whenever the Python-bindings for the USRP are requested. BorIP automatically patches into usrp_source_c and will attempt to connect to a BorIP server if no local device is present. The default server address is specified in ~/.gnuradio/config.conf:

[borip]
server=<server address>

The other settings are:

reconnect_attempts=<# of reconnection attempts before signalling EOF to the flowgraph, -1 to try forever, default is 0>
reconnect_interval=<seconds between connection attempts, default 5 seconds>
keepalive_interval=<seconds between sending PING keepalive, default is 5 seconds>
verbose={True,False} Verbose mode prints commands sent between client and server
default_port=<TCP and UDP port to use, default is 28888>

If you use the BorIP Source GRC block, the generated code will NOT honour the reconnect_attempts specified in the above config file. Instead it will always take it from the parameter set for the GRC block instance.

auto_fec

Automatically try every combination of FEC parameters until the correct ones are found (determined by monitoring BER output by Viterbi decoder).

This uses the NASA Voyager (k=7) convolutional code.

Auto FEC block (MPSK symbols in, deFEC'd data out with measure of BER and FEC-lock signal

Leave 'sample rate' parameter at 0 to have it interpret duration/periods as number of samples (this should work fine in all cases anyway). If you specify the sample rate, you can optionally engage the internal Throttle if playing back from a file.

eye

This draws upon the Data Scope originally by Max (KA1RBI) from OP25. The block allows for changing certain sampling/graphical parameters at runtime (see GRC block definition below).

Eye diagram showing 2-level signal

facsink

Please see Fast Auto-correlation.

Fast Auto-correlation block at bottom-right

grc

GRC XML block definitions for:

rtl_source_c: RTL2832 Source
agc: AGC (Baz) - this is unfinished
auto_fec: Auto FEC
borip_source: BorIP Source
delay: Variable Delay
puncture_bb/depuncture_ff: Puncture/Depuncture
pow_cc: Power
swap_ff: Swap
test_counter_cc: Test Counter
eye: Eye Diagram
facsink: Fast AutoCorrelation Sink
op25: OP25 Decoder
- The Key parameter will only work if the decryption patch is applied to the OP25 code. (Otherwise it should be left blank.)
message_callback: Message Callback feedback mechanism
gr_mpsk_receiver_debug_cc: MPSK Receiver (Debug)
- Requires appropriate patch to gr_mpsk_receiver_debug.cc

Also, this contains patches for GRC to enable 'any' block support. This enables you to use raw GNU Radio blocks by typing in the necessary Python function to create the block (without a dedicated GRC XML block definition). For example: I use it with baz.print_char

baz_any_source: Any Block Source (for source blocks)
baz_any_sink: Any Block Sink (for sink blocks)
baz_any: Any Block (for blocks with I/O)

'Any' block support is now in the latest official GRC release, so you don't have to apply the Python code patch (but you will still need the XML block definitions above).

If you don't have the latest GRC, you can either apply the following two patches, OR apply one from the patch page: