My Anytone 878UV Codeplug

A declarative approach to codeplug programming

I’m struggling with editing my codeplug in CPS, here is a little experiment to see if I can handle my codeplug using some Emacs Org Mode magic.

1 Introduction

Will it be possible to handle a codeplug using just a text file? Here in this page I’ll give it a try.

With Emacs org-mode and literate programming I can put my software, database, codeplug, and documentation in just a single file. If you are reading this page on the web, this is just and automated export of my single org file.

1.1 Architecture

The goal here is to use sqlite ability to read and write CSV files to dinamically generate a codeplug using an export from IZ3WNH Repeater Map.

This document generates a temporary database, import data from external CSV files as well as from the tables defined in this document, and finally export a partial CSV codeplug for my Anytone radio.

2 The database

This section contains the database schema, the main takeaway here is that we are implementing autoincrementing keys so that we don’t have to deal with memories numbers manually.

drop table if exists radio_id_list;
drop table if exists channels;
drop table if exists talk_groups;

create table radio_id_list
(
       "No." integer not null
             constraint radio_id_list
             primary key autoincrement,
       "Radio ID" TEXT,
       Name TEXT
);

create unique index "radio_id_list_No._uindex"
         on radio_id_list ("No.");

create table talk_groups
(
       "No." integer not null
             constraint talk_groups
             primary key autoincrement,
       "Radio ID" TEXT,
       Name TEXT,
       "Call Type" TEXT,
       "Call Alert" TEXT
);

create unique index "talk_groups_No._uindex"
         on talk_groups ("No.");
create unique index "talk_groups_name._uindex"
         on talk_groups (Name);

drop table if exists channels;
create table channels
(
    "No." integer not null
          constraint channels
          primary key autoincrement,
    "Channel Name"                 TEXT,
    "Receive Frequency"            TEXT,
    "Transmit Frequency"           TEXT,
    "Channel Type"                 TEXT,
    "Transmit Power"               TEXT,
    "Band Width"                   TEXT,
    "CTCSS/DCS Decode"             TEXT default 'Off',
    "CTCSS/DCS Encode"             TEXT default 'Off',
    Contact                        TEXT,
    "Contact Call Type"            TEXT,
    "Contact TG/DMR ID"            TEXT,
    "Radio ID"                     TEXT,
    "Busy Lock/TX Permit"          TEXT,
    "Squelch Mode"                 TEXT default 'Carrier',
    "Optional Signal"              TEXT default 'Off',
    "DTMF ID"                      TEXT default '1',
    "2Tone ID"                     TEXT default '1',
    "5Tone ID"                     TEXT default '1',
    "PTT ID"                       TEXT default 'Off',
    "Color Code"                   TEXT default '1',
    Slot                           TEXT default '2',
    "Scan List"                    TEXT default 'None',
    "Receive Group List"           TEXT default 'None',
    "PTT Prohibit"                 TEXT default 'Off',
    Reverse                        TEXT default 'Off',
    "Simplex TDMA"                 TEXT default 'Off',
    "Slot Suit"                    TEXT default 'Off',
    "AES Digital Encryption"       TEXT default 'Normal Encryption',
    "Digital Encryption"           TEXT default 'Off',
    "Call Confirmation"            TEXT default 'Off',
    "Talk Around(Simplex)"         TEXT default 'Off',
    "Work Alone"                   TEXT default 'Off',
    "Custom CTCSS"                 TEXT default '2511.0',
    "2TONE Decode"                 TEXT default '1',
    Ranging                        TEXT default 'Off',
    "Through Mode"                 TEXT default 'Off',
    "APRS RX"                      TEXT default 'Off',
    "Analog APRS PTT Mode"         TEXT default 'Off',
    "Digital APRS PTT Mode"        TEXT default 'Off',
    "APRS Report Type"             TEXT default 'Off',
    "Digital APRS Report Channel"  TEXT default '1',
    "Correct Frequency[Hz]"        TEXT default '0',
    "SMS Confirmation"             TEXT default 'On',
    "Exclude channel from roaming" TEXT default '0',
    "DMR MODE"                     TEXT,
    "DataACK Disable"              TEXT default '0',
    R5toneBot                      TEXT default '0',
    R5ToneEot                      TEXT default '0',
    "Auto Scan"                    TEXT default '0',
    "Ana Aprs Mute"                TEXT default '0'
);

create unique index "channels_No._uindex"
         on channels ("No.");

drop table if exists zones;
create table zones
(
    "No." integer not null
          constraint zones
          primary key autoincrement,
    "Zone Name"                        TEXT,
    "Zone Channel Member"              TEXT,
    "Zone Channel Member RX Frequency" TEXT,
    "Zone Channel Member TX Frequency" TEXT,
    "A Channel"                        TEXT,
    "A Channel RX Frequency"           TEXT,
    "A Channel TX Frequency"           TEXT,
    "B Channel"                        TEXT,
    "B Channel RX Frequency"           TEXT,
    "B Channel TX Frequency"           TEXT
);

create unique index "zones_No._uindex"
         on zones ("No.");

3 DMR radio ID

Each DMR radio requires a DMR ID, here is mine, you must get your on RadioID.

Radio ID Name
2227589 IU5BON / Alessio

With the following piece of code, we record this table into the codeplug.

create temporary table temp_table("Radio ID" text, Name text);
.mode csv temp_table
.import $mydata temp_table

insert into radio_id_list ("Radio ID", Name) select * from temp_table;

4 Talk Groups

This is the list of talkgroups that I like to have on my radio

Radio ID Name Call Type Call Alert
2241 TOSCANA MP Group Call None
22292 ITA MP Group Call None
92 EU Group Call None
91 WW Group Call None
9 Local Group Call None
99 Simplex Group Call None
9999996 Hotspot OFF Private Call None
9999998 Hotspot Reboot Private Call None
4000 Unlink Group Call None
2620311 Andreas Private Call None
222999 APRS Private Call None
9990 Parrot Private Call None
222907 JOTA IT Group Call None
2225098 I5EKX Alex Private Call None
98 Radio Test Group Call None
973 SOTA Group Call None
3181 POTA Group Call None
31001 Net Talkgroup 1 Group Call None
31002 Net Talkgroup 2 Group Call None
222001 TAC1-ITA Group Call None
222002 TAC2-ITA Group Call None
222003 TAC3-ITA Group Call None
222004 TAC4-ITA Group Call None
222005 TAC5-ITA Group Call None
222006 TAC6-ITA Group Call None
222007 TAC7-ITA Group Call None
222008 TAC8-ITA Group Call None
222009 TAC9-ITA Group Call None
222010 TAC10-ITA Group Call None
222555 Cluster GRF Group Call None
65000 XLX Status Private Call None
64000 XLX Disconnect Private Call None
6 XLX Group Call None
64001 XLX Module A Private Call None
64002 XLX Module B Private Call None
64003 XLX Module C Private Call None
64004 XLX Module D Private Call None
64005 XLX Module E Private Call None
64006 XLX Module F Private Call None

As usual, we use the above table to generate the codeplug.

.mode csv

drop table if exists my_talk_groups;
.import $mydata my_talk_groups

insert into talk_groups ("Radio ID", Name, "Call Type", "Call Alert")
       select "Radio ID", Name, "Call Type", "Call Alert" from my_talk_groups;

5 Zones - Channels

This radio is organized with channels and zones, a channel may belong to multiple zones, but to simplify the implementation we describe our codeplug zone by zone, each one with its own channels.

To automatically generate the zone entry we can use this function that generate the zone for us.

NOTE: The first channel will be the default for VFO A and the second will be the default for VFO B.

-- func generate_zone(name, channels)

with a_channel as (select "Channel Name", "Receive Frequency", "Transmit Frequency"
                   from $channels
                   limit 1),
     b_channel as (select "Channel Name", "Receive Frequency", "Transmit Frequency"
                   from $channels
                   limit 1 offset 1)
insert
into zones("Zone Name", "Zone Channel Member", "Zone Channel Member RX Frequency",
           "Zone Channel Member TX Frequency", "A Channel", "A Channel RX Frequency", "A Channel TX Frequency",
           "B Channel", "B Channel RX Frequency", "B Channel TX Frequency")
select '$name',
       group_concat(c."Channel Name", ' | '),
       group_concat(c."Receive Frequency", ' | '),
       group_concat(c."Transmit Frequency", ' | '),
       a_channel."Channel Name",
       a_channel."Receive Frequency",
       a_channel."Transmit Frequency",
       b_channel."Channel Name",
       b_channel."Receive Frequency",
       b_channel."Transmit Frequency"
from $channels as c,
     a_channel,
     b_channel
order by c."Channel Name";

-- end func

5.1 Hotspot zone

I want an hotspot zone with all my TGs and the following parameters.

Freq. 433.675
Tx Power Low
APRS RX Off

Here we generate the channels.

insert into channels ("Channel Name", "Receive Frequency", "Transmit Frequency", "Channel Type", "Transmit Power",
                      "Band Width", Contact, "Contact Call Type", "Contact TG/DMR ID",
                      "Busy Lock/TX Permit", "APRS RX", "DMR MODE")
select Name,
       '$freq',
       '$freq',
       'D-Digital',
       '$tx_power',
       '12.5K',
       Name,
       "Call Type",
       "Radio ID", -- TG or Private ID
       'Same Color Code',
       '$aprs_rx',
       '1'
from talk_groups;

And with generate_zone(name=’Hotspot’, channels=’channels’) we pack all the new channels into an Hotspot zone.

5.2 Simplex zone

Channel Name Receive Frequency Transmit Frequency Channel Type Transmit Power Band Width CTCSS/DCS Decode CTCSS/DCS Encode Contact Busy Lock/TX Permit APRS RX Analog APRS PTT Mode APRS Report Type SMS Confirmation DMR MODE
ARI FI 145.225 145.225 A-Analog High 25K Off Off Local Off Off Off Off Off 0
Call VHF 145.5 145.5 A-Analog High 25K Off Off Local Off Off Off Off Off 0
Call UHF 433.5 433.5 A-Analog High 25K Off Off Local Off Off Off Off Off 0
APRS 144.8 144.8 A-Analog High 25K Off Off Local Off Off Start Of Transmission Analog Off 0
APRS VA 144.8 144.8 A-Analog High 25K 136.5 136.5 Local Off Off End Of Transmission Analog Off 0
DMR Call V 145.45 145.45 D-Digital High 12.5K Off Off Simplex Always On Off Off Off 0
DMR Call U 433.45 433.45 D-Digital High 12.5K Off Off Simplex Always On Off Off Off 0

We convert the above table into our channels.

.mode csv
drop table if exists my_simplex_channels;
.import $simplex my_simplex_channels

-- channels
insert into channels ("Channel Name", "Receive Frequency", "Transmit Frequency", "Channel Type", "Transmit Power",
                      "Band Width", Contact, "Contact Call Type", "Contact TG/DMR ID",
                      "Busy Lock/TX Permit", "APRS RX", "DMR MODE")
select "Channel Name",
       "Receive Frequency",
       "Transmit Frequency",
       "Channel Type",
       "Transmit Power",
       "Band Width",
       Contact,
       tg."Call Type",
       tg."Radio ID",
       "Busy Lock/TX Permit",
       "APRS RX",
       "DMR MODE"
from my_simplex_channels as c
         join talk_groups as tg on tg.Name == c.Contact;

And we pack all the new channels into a Simplex zone.

5.2.1 TODO Allow filtering TGs during channel generation

5.3 Import national repeaters

In order to complete my codeplug I need an export from IZ3WNH Repeater Map.

Export instructions:

  1. Visit IZ3WNH Repeater Map
  2. Click the top left funnel icon
  3. Filter by:
    • Country: Italia
    • Type::
      • DMR
      • FM
      • Echolink
      • C4FM these repeaters also work on FM
    • Band: 144MHz and 433MHz
  4. Click the SEARCH button
  5. Fill the second section of the form:
    • Memory Name Format: Callsign
    • Optional Format: County + DMR Network
    • Select Decimal Separator: Dot
    • Select Radio Model: AnyTone - D878UV CPS v1.24N
  6. Click Download memories only
  7. Extract the downloaded zip file into source/import

Now we can import them, updating the contact and the time slot.

.mode csv
drop table if exists my_imported_channels;
.import source/import/01_Anytone_D878UVv124N_D878UVIIv202_Channel.CSV my_imported_channels
drop table if exists my_imported_zones;
.import source/import/03_Anytone_D878UVv124N_D878UVIIv202_Zone.CSV my_imported_zones

-- set a default Contact

update my_imported_channels
    set Contact = 'Local',
        "Contact TG/DMR ID" = '9',
        Slot = '2';

-- bulk import channels

insert into channels("Channel Name", "Receive Frequency", "Transmit Frequency", "Channel Type", "Transmit Power",
                     "Band Width", "CTCSS/DCS Decode", "CTCSS/DCS Encode", "Contact", "Contact Call Type",
                     "Contact TG/DMR ID", "Radio ID", "Busy Lock/TX Permit", "Squelch Mode", "Optional Signal",
                     "DTMF ID", "2Tone ID", "5Tone ID", "PTT ID", "Color Code", "Slot", "Scan List",
                     "Receive Group List", "PTT Prohibit", "Reverse", "Simplex TDMA", "Slot Suit",
                     "AES Digital Encryption", "Digital Encryption", "Call Confirmation", "Talk Around(Simplex)",
                     "Work Alone", "Custom CTCSS", "2TONE Decode", "Ranging", "Through Mode", "APRS RX",
                     "Analog APRS PTT Mode", "Digital APRS PTT Mode", "APRS Report Type", "Digital APRS Report Channel",
                     "Correct Frequency[Hz]", "SMS Confirmation", "Exclude channel from roaming", "DMR MODE",
                     "DataACK Disable", "R5toneBot", "R5ToneEot", "Auto Scan", "Ana Aprs Mute")
select "Channel Name",
       "Receive Frequency",
       "Transmit Frequency",
       "Channel Type",
       "Transmit Power",
       "Band Width",
       "CTCSS/DCS Decode",
       "CTCSS/DCS Encode",
       "Contact",
       "Contact Call Type",
       "Contact TG/DMR ID",
       "Radio ID",
       "Busy Lock/TX Permit",
       "Squelch Mode",
       "Optional Signal",
       "DTMF ID",
       "2Tone ID",
       "5Tone ID",
       "PTT ID",
       "Color Code",
       "Slot",
       "Scan List",
       "Receive Group List",
       "PTT Prohibit",
       "Reverse",
       "Simplex TDMA",
       "Slot Suit",
       "AES Digital Encryption",
       "Digital Encryption",
       "Call Confirmation",
       "Talk Around(Simplex)",
       "Work Alone",
       "Custom CTCSS",
       "2TONE Decode",
       "Ranging",
       "Through Mode",
       "APRS RX",
       "Analog APRS PTT Mode",
       "Digital APRS PTT Mode",
       "APRS Report Type",
       "Digital APRS Report Channel",
       "Correct Frequency[Hz]",
       "SMS Confirmation",
       "Exclude channel from roaming",
       "DMR MODE",
       "DataACK Disable",
       "R5toneBot",
       "R5ToneEot",
       "Auto Scan",
       "Ana Aprs Mute"
from my_imported_channels
order by "Channel Name";

-- bulk import zones

insert into zones("Zone Name", "Zone Channel Member", "Zone Channel Member RX Frequency",
                  "Zone Channel Member TX Frequency", "A Channel", "A Channel RX Frequency", "A Channel TX Frequency",
                  "B Channel", "B Channel RX Frequency", "B Channel TX Frequency")
select "Zone Name",
       "Zone Channel Member",
       "Zone Channel Member RX Frequency",
       "Zone Channel Member TX Frequency",
       "A Channel",
       "A Channel RX Frequency",
       "A Channel TX Frequency",
       "B Channel",
       "B Channel RX Frequency",
       "B Channel TX Frequency"
from my_imported_zones
order by "Zone Name";

6 Codeplug generation

Now that everything is in place, it’s time to export the codeplug database into several CSV files compatible with Anytone’s CPS.

-- set default radio id
UPDATE channels
       SET "Radio ID" = '$default_radio_id'
       WHERE "Radio ID" is NULL or "Radio ID" not in (select distinct Name from radio_id_list);

.mode csv
.headers on

.output codeplug/RadioIDList.CSV
select * from radio_id_list;

.output codeplug/TalkGroups.CSV
select * from talk_groups;

.output codeplug/Channel.CSV
select * from channels;

.output codeplug/Zone.CSV
select * from zones;

Congratulations for making so far in this page!

If you found the above document challanging here comes the trickiest part. It is absolutely fine to stop reading here, what come next is really a technical detail.

6.1 Post-Processing

sqlite3 exports CSV files with LF (unix style) but Anytone’s CPS wants it in CRLF (windows style). Here we make sure generated files are converted with the proper line ending encoding.

(defun convert-file-to-crlf (@fpath)
  "Convert file's encoding.
*fpath is full path to file.

Based on `http://ergoemacs.org/emacs/elisp_convert_line_ending.html'"
  (let ($buffer
        ($bufferOpened-p (get-file-buffer @fpath)))
    (if $bufferOpened-p
        (with-current-buffer $bufferOpened-p
          (set-buffer-file-coding-system 'dos)
          (save-buffer))
      (progn
        (setq $buffer (find-file @fpath))
        (set-buffer-file-coding-system 'dos)
        (save-buffer)
        (kill-buffer $buffer)))))

;; loop over CSV files in the codeplug folder and convert them
(dolist (file (directory-files "codeplug" t "\.CSV$"))
  ;; skip the DMR ID database
  (unless (string-equal "DigitalContactList" (file-name-base file))
    (convert-file-to-crlf file)))

6.2 codeplug-diff tool

Checking codeplug changes can be tricky with diff, to make my life easier I use this script based on cvsdiff.

set -q DIFFOPTS || set DIFFOPTS -o color-words

for file in (git status --porcelain codeplug/*.CSV | awk '{print $2}')
    echo *********** Changes in $file \n
    csvdiff $DIFFOPTS (git show HEAD:$file | psub) $file
end

To install csvdiff on MacOS use the following command.

brew install thecasualcoder/stable/csvdiff