    TITLE=todo DESCRIPTION=to-do list for remstats development
    KEYWORDS=todo DOCTOP=index DOCPREV=bugs DOCNEXT=faq # Last is
    134

To-Do List for Remstats
-------------------------------------------------------------------

High Priority
    134 20010829 [LOW] - make header_bar (in htmlstuff) do the link
    making, if available and fix whatever uses it not to.

    133 20010829 [LOW] - add an option to make nt-discover update
    old hosts with a standard set of RRDs, even if the hosts are
    already known

    132 20010824 [HIGH] - BUG: get rid of the spikes in uptime from
    the unix-status-server

    131 20010824 [MED] - make status pages for each host, group and
    for all hosts using the new alertstatus and possibly alertvalue.

    130 20010823 [HIGH] - add an <RRD::EXEC ...> tag to rrgcgi. To
    by used in host index pages (see 129).

    128 20010629 [MED,HOLD] - custom, configuration-supplied info
    per rrd which is simply available wherever it makes sense, e.g.
    in alerts.

    - first make sure someone has a use for it.

    127 20010622 [MED] - graph data together with historical data.
    This will probably mean either populating another rrd with
    historical averages, temporarily or permanently, or modifying
    rrdtool. The former is certainly simpler to do, given my
    knowledge of the internals of rrdtool. However, it needs to have
    another rrd for each period? Need to keep the same data over
    some longer period, a multiple of the period of interest, as
    well as the averages, from period to period.

    122 20010330 [HIGH] - rrd prog-* which tells if a particular
    named process is running, using the ps section of the unix-
    status-collector.

    121 20010202 [MED,HOLD] - how about an discovery program, to
    find and identify hosts and then run the appropriate new-xxx-
    hosts scripts to add them?

    DONE 20010608 - nt-discover to find and add NT boxen

    115) 20001229 [HIGH] - need docs on errors. Specifically, when
    run-remstats kills a collector for taking too long. And where to
    find the output of the killed collector.

    112) 20001212 [LOW,HOLD] - web-based remstats configurator.
    Needs to consider security, at least from the point of view that
    you don't want to lose your configuration. The most important
    part is hosts. A lot of the rest doesn't have to be changed, or
    only once.

    111) 20001212 [LOW,HOLD] consider grafting on (at least links
    to) some kind of system configuration interface. For configuring
    the mmonitored entities, not remstats.

    110) 20001212 [LOW,HOLD] consider problem-fixing interface. It'd
    be nice to try to fix things if there is a known way to do so. A
    simple kludge would be to add another method to the alert-
    destination-map which deals with problems that it knows about,
    possibly invoking plugins for specific alerts.

    109) 20001212 [MED] nt-log-collector, with modules for event-
    logs and ntmail logs.

    70) 20000407 [HIGH] CGI scripts need to have a way to deal with
    alternate config-files, and graph-writer needs to tell them if
    they can't work it out themselves. Otherwise, people need to be
    told to do multiple installs of the CGI scripts, which might be
    the best way.

            make install-cgis CONFIGDIR=config-xxx

    Not that painfull, but wastefull and makes upgrade messier.

    - I don't like the multiple-install method, but any other method
    needs a way of getting configuration information into the CGI
    scripts. Any method which passes info in via the URL or form
    fields is out: too unsafe. The only other method I can think of
    is to read a configuration file in the same directory as the CGI
    script. This ought to be safe from modification, or your web-
    site is waiting to be mutilated. The other part to consider is
    whether any part of the info in the CGI config-file is
    sensitive. I.E. do we have to protect it in some way.

    - Configuration file in the same directory won't work either,
    you'd still have to install the cgi's multiple times. I'm
    starting to think that multiple installations may be the only
    safe thing to do.

    99) 20000619 [HIGH] make unix-status-collector send the
    directories that we want df for and make unix-status-server do
    "df /dir1 /dir2" to get them, and pull them off one line at at
    time. This is to deal with things like disconnected NFS-mounted
    directories hanging df when we do just a bare "df".

    86) 20000419 [HIGH] trends analysis

    87) 20000419 [HIGH] alerts based on trends analysis and
    historical data, like one-week average and standard-deviation,
    ... (for Steve)

    106) 20000922 [MEDIUM] make a file-collector. Similar to the
    log-collector, only for small, local files. Slurp the file into
    memory, match patterns and pull out values. The data line in an
    rrd definition would be like:

            source file
            data VARNAME    GAUGE:600:0:U FUNCTION PATTERN(WITH)PARENS

    in fact, this would share so much code with the log-collector
    that it might be worth combining the two. This allows collection
    from things like Linux's /proc.

    98) 20000619 [MEDIUM] add group index files and store hosts
    under group directories. For easier application of access-
    controls. (for Florian)

    2) ???????? [MEDIUM-INPROGRESS] make rrd munger, like copyrrd
    was supposed to be use dump/restore and process the xml form
    (rrddump-munger) what functions do we need? Make one script for
    each function.

    add a DS (less important, as we can just make a new rrd)
    remove a DS (less important, as we can ignore it)
    add an archive
    extend an archive
    change CF of an archive
    remove an archive
    filter data within an archive
    change NaN to number/max/min
    change ># to NaN/max/min
    change <# to NaN/max/min
-------------------------------------------------------------------

Lower Priority
    102) 20000912 [LOW] add see-also to host config, which will
    materialize links in the host header. Config line like:

            seealso host:xyzzy http://www.somewhere ftp://ftphost

    the special "host:" pseudo-URL gets changed to a link to the
    remstats page for that host.

    103) 20000915 [MEDIUM] make-path doesn't work with non fqdn
    hosts Make it read the configuration, so it can look up the IP
    number in the host config and use that if it's defined.
    Otherwise, default to gethostbyaddr.

    107) 20000922 [MEDIUM] extra status header lines for hosts, from
    specified STATUS files creaded by the various collectors. Add
    lines to host definition like:

            extrastatus "STATUS DESCRIPTION" STATUS-FILE-NAME

    60) 20000328 [MEDIUM] replace route-collector with something
    which scales. SNMPwalking bgp4PathAttrBest doesn't scale to
    large Internet routers with 400 peers, taking over an hour to
    complete. (see also 61)

    - look at a script to follow the output of zebra. That's a lot
    of overhead though. Easy if zebra is solid.

    - How difficult can it be to make a native BGP listener? I'm not
    clear on the protocol, but it doesn't look too bad.

    45) 20000121 [MEDIUM] make snmp-collector send only one packet
    per host

    - test and make sure that we do get back whatever succeeded. I
    vaguely remember that it didn't work. [Later: at least under UCD
    snmp under linux, if an item isn't implemented in the MIB, you
    get back NOTHING. Specifically, look for the non-unicast packet
    counters as well as something else; you get nothing back. This
    isn't good.]

    - have to re-write snmp-collector completely, which isn't that
    bad an idea. This means a two-pass structure. On pass one, we
    construct the complete query and then send it. On pass two, we
    examine all the results and format them.

    9) ???????? [MEDIUM-TESTING] make alerts take connectivity
    dependence into account

    - add "via" line to host section to deal with hubs and switches
    [DONE]

    - I think it's done. See what happens next outage.

    42) 20000114 [MEDIUM] snmp-collector mod to allow summary data
    collected from a walk and then filtered as a single data-point.
    E.G. specify a rrd "oid" like:

            walk    count ifOperStatus = 1

    would produce a count of the number of interfaces on that device
    that were active (i.e. had a live device plugged into them). Or
    a similar one would let you count BGP routes, or arp addresses,
    ...

    - Unfortunately, from experience with the snmp-route-collector,
    this is going to be slow for anything with a large number of
    items.

    43) 20000114 [MEDIUM] parallelizing the collectors, at least on
    a group basis, preferably host or group.

    - collectors must accept `-G' and `-H' flags to request
    processing of the specified group or host, respectively. Run-
    remstats needs to fork extra processes according to a config-
    file line, "parallel group" or "parallel host".

    - 20010831 TEE - implemented -H flags for all collectors except
    for the remoteping-collector, which I'm not using anyway right
    now.

    51) 20000216 [LOW] need a way to specify URL for port-http. The
    root page doesn't always exist.

    37) 19991216 [LOW] traceroute sometimes shows incorrect routing,
    which confuses the topology-monitor, causing false positives

    50) 20000215 [LOW] make inventory script. Runs uname (for
    hardware and software), `ifconfig -a', `netstat -nr', `hostname'
    and any others I can think of to collect configuration info.
    Then figures out the versions of important software, e.g. run
    `perl -v', `gcc -v ...' Make a subdir to put it in and make a
    tool definition to get it onto the host pages.

    - looks like the beginning of a discovery script.

    62) 20000329 [LOW] make different markers for different levels
    of alert on quick-index.

    69) 20000406 [LOW] is there any use for write_environment in
    check-config?

    97) 20000616 [LOW] make port-collector or check-config complain
    about having a script with ok/warn/error/critical patterns but
    no send string. The port-collector will ignore patterns unless
    there is a send string.

-------------------------------------------------------------------

On Hold
    Usually waiting for next major release, or trapped by something
    else. (in priority order)

    40) 20000104 [MEDIUM-HOLD] consider some form of access-control
    for servers

    - hash-based "password"

    - ssl tunneling ought to work for everything except SNMP

    - what does this buy? With the various servers run under
    tcp_wrappers an attacker must either gain access to the remstats
    collector machine or spoof a tcp session from them. If you've
    been "owned" you've got bigger problems. If the attacker spoofs
    a session with a remstats server, tcp-wrappers will insist that
    it must come from one of the allowed hosts, so that's where the
    stolen output will go. This is only usefull to the attacker if
    they have access to the remstats collector machine or if they
    can sniff the traffic between the collector and the server. The
    only data loss possible is with the log-server which keeps
    state. (Ignoring DOS attacks which are always a problem.)

    - unless someone needs this, it's on hold

    6) ???????? [LOW-NEEDS:2-HOLD] increase CA3 resolution

    - need rrd munger (2)

    10) ???????? [LOW-INPROGRESS-HOLD] make graph of connectivity

    13) ???????? [LOW-INPROGRESS-HOLD] snmp trap listener to update
    status files

    - needs filter to be usefull [DONE]

    - I haven't seen any useful traps so this is on hold.

    14) ???????? [LOW-NEEDS:2-HOLD] make rrd structural changes in
    config file get applied to the rrds.

    - some taken care of with snmpif-setspeed, but need a more
    general solution

    - look at new XML output of rrddump

    39) ???????? [LOW-HOLD] make RRD dumper, to put data out in a
    form that can be loaded into a database

    - I don't need it, per se, but it might be easier than writing
    the availability report generator.

    52) 20000215 [LOW] make a makegraph.cgi, or whatever, that will
    let you make a somewhat custom graph on the fly. makegraph.cgi
    by itself will list all the hosts and let you choose one.
    makegraph.cgi?host=xxx will list all the RRDs for this host and
    let you choose ?one?. makegraph.cgi?host=xxx&rrd=yyy will list
    the various DSs for this RRD and let you choose the ones you
    want. Then you get to define any CDEFs needed and then
    LINEn/AREA/STACK for each DEF or CDEF desired. And size, title,
    legends...

    - On hold since graph.cgi will let you get at any existing graph
    you want. If I find a use or need for this, I'll re-activate it.

    92) 20000518 [HIGH] collect traffic info from cflowd
    (artsportms). Make it flexible enough that it can let you choose
    which ports you want (one per rrd?). Make a loader to load
    historical data.

    - [DONE 20000524] artsportms-loader done

    - I no longer have access to devices with this feature

-------------------------------------------------------------------

    I've also kept the stuff that used to be here, but has already
    been done.

------------------------------------------------------------------
Last updated Mon Sep 10 15:18:26 EDT 2001 by <thomas.erskine@sourceworks.com>.

