Connectionist Networks

Sean Finney

Yoshi Komori

Mark Angelillo

CS63 Fall 2002


Neural Net Architecture

What could be better than a fully connected network? Using the xtlearn program, we were able to analyze different types of networks, using a fully connected network as a benchmark with which we could gauge the speed and learning ability of the different networks. By implementing feature detectors, we were able to create a more efficient network than one that is fully connected. The most defining areas or inputs of each possible output are the "features" of the outputs. By having the network pay attention to these features, the desired outputs will be learned faster.

Digits

The network is trying to learn to recognize the difference between the 10 digits. We give it a 6 by 6 grid of 0's or 1's that are arranged such that the 1's trace out the digit. The different digits have different places where 1's will tend to cluster. For example, a 1 tends to dominate one of the middle rows. These are the features we are trying to define. Once defined, these features will allow the network to pin down what digit it is looking at.

Network Models

While searching for a method with which we could solve this problem, we tried a few different networks. Originally, we tried using 10 hidden nodes, using 6 of them to watch the rows of the grid, and the other 4 watched the corners. This performed horribly. Then we decided to get smart. We thought about the spots on the digits that would differ from number to number, and assigned the 10 hidden nodes to our decided pertinent spots on the grid. This is the "guess" implementation that appears in our tables. It performed better, but not by much. We decided to get smarter still. What do all the numbers have for their most important input nodes? Why, the nodes that appear most often in the data, of course. So we wrote a nifty program to create a two- dimensional histogram of the hits on input nodes for each digit. Then, for each digit, a new histogram is created by subtracting the number of hits each other digit got on an input node from the value on that input node for that digit. Each of the 10 hidden nodes in this new network has, as its input nodes, the 10 greatest valued nodes on that final histogram. This method performed better, but we still weren't getting the kind of results we wanted. We then decided to take not just the 10 highest valued nodes, but the 9 highest valued nodes, and the 6 lowest valued as well. The idea here is that the lowest values are just as defining as the highest. This is the "highlow" implementation in the tables. This performs better still, but the problem here is that some of the input nodes are never being looked at. Finally, we added 5 nodes that were fully connected so the network would always be seeing the entire grid. This is the "best" implementation. We found that this method performed much better than any of the others.

Data Analysis

In order to analize the effectiveness of different networks, it is necessary to compute the number of connections each network has.

Fully Connected: 36(input) * 10(hidden) + 10(hidden) * 10(output) + 20(bias) = 480
Guess: 6(input) * 10(hidden) + 10(hidden) * 10(output) + 20(bias) = 180
Highlow: (6(high) + 9(low)) * 10(hidden) + 10(hidden) * 10(output) + 20(bias) = 270
Best: (6(high) + 9(low)) * 10(hidden) + 36(input) * 5(hidden) + 10 (hidden) + 10(hidden) * 10(output) + 25(bias) = 455

In general, we predict that the more connections a network has, the longer it takes to learn and the more correct its prediction becomes once it has learnt. This general tendency did not hold for our experiment. For experiments with 200,000 training sweeps, it was true that the networks with more connections, the fully connected and the best, performed significantly better. However, those same networks also performed better at 25,000 training sweeps, suggesting that they also learn faster than the other two. As we have mentioned before, this can be attributed to the fact that the guess and the highlow do not have access to all the input data. For each digit, they have access only to 6 or 15 input nodes, which turned out to be insufficient to deal with idiosyncratic representations of digits people came up with.

The more interesting comparison is between the fully connected and the best. Even though the best had less connections, it performed marginally better at both the 25,000 and 200,000 training sweeps. We believe that its success is due to its architecture that overcame the weakness of the highlow network. Even though the highlow did not perform as well as the fully connected, it was able to predict 80% to 90% of the data. This suggests that the features we extracted was indeed a good indication of the digits. All it was missing was a way to identify the small portion of the data that could not be distinguished from the features. The best network, as 5 of its hidden nodes were fully connected to the input nodes, was able to access and analyze data that dealt with those idiosyncratic digits. Consequently, the best network combined the strength of both the fully connected and the highlow networks. It had the efficiency of the highlow network: it could identify the major part of digits with small number of connections. And it also had the robustness of the fully connected network by having access to all the data. The combined effect was its performance more successful than the fully connected, with fewer connections than the latter.

For cluster analysis, we found it convenient to compress the data by expressing each cluster as a single digit. With this compression, we map the trees into sequences of digits that look like:

Guess = 10216427285873726394
Highlow = 416548083279
Full = 08054864146327379
Best = 8054612379

There are many interesting features we can extract out of the cluster analysis. First, we see certain similarities across all networks. For instance, 7 is clustered close to 9, and 0 is clustered close to 8. This tendency corresponds to the similarity across how digits look. If we look at digits, 7 does look similar to 9 and 0 looks similar to 8. Thus the clusters indicate that the networks were distinguishing digits in a way perhaps similar to how humans distinguish them.

Second, we see that the guess and the fully connected networks splitted each digits into more clusters than the highlow and the best networks did. For the guess, this reflects the fact that the network was not understanding the digits in a coherent manner. However, it is interesting to consider why the fully connected network had a messier clustere than the highlow network, even though the former performed significantly better than the latter did. Perhaps we can consider the splitting of a digit into more than one cluster as a lack of generalization. For example, for the fully connected network, the digit 0 is split into two clusters. We can understand this as that the network could not generalize many instances of 0 into a single concept 0, but instead had to come up with two concepts of 0. If we thought of the clusters in this manner, that each cluster is a generalization of instances to a single concept, then we can deduce that the highlow was better at generalization than the fully connected. On the other hand, if we looked at the original trees more carefully, we find a lot more misplaced digits in the highlow network than the fully connected one. This fact can perhaps be interpreted as mistaken generalization. The highlow tried to generalize more than the fully connected tried, but due to the lack of information made many mistakes in that process.

Now, let us look at the cluster of the best network. Based on the previous anthropomorphizing of the clusters, we see that the best network generalized digits in just the right manner while making very few mistakes. The best network came up with a single concept for each digit, as it should.

Last, we are perplexed and intrigued by the comparison between the best and the fully connected networks' clusters. If we looked at the trees, it is clear that the one of the best network is significantly better than the one of the fully connected network: it had the right generalization with fewer errors. On the other hand, we only see a marginal difference (1%) in terms of performance. If the best network had an understanding of the digits so much greater than that of the fully connected, why didn't it perform a lot better, too? We do not have a confident answer to this question, but we suspect that the answer lies within the connections between the hidden nodes and the output nodes. Even though the fully connected network had a flawed understanding of the input, it perhaps somehow dealt with the imperfections within the connections between the hidden nodes and the output nodes. If we may use our humble example of robot soccer, we can make an analogy to the subsumption architecture. In our implementation of soccer playing agents, the localization module receives data from the vision module, and tries to estimate where the agent is. In this analogy, the raw data the camera recieves is the input nodes, the vision module is the connections between the input nodes to the hidden nodes, the data that the vision module transfers to the localization module are the hidden nodes, the localization module is the connections between the hidden nodes to the output nodes, and the output nodes are the final estimate of its location. So the best network is analogous to an agent that has a superb vision module. The vision module is so good that the localization module receives almost perfect data, and consequently the localization doesn't have to do much work. On the other hand, the fully connected network is where the vision module isn't that good. The data the localization module receives have many errors. However, the localization module is smart enough to deal with the errors in data. It performs particle filters and other techniques to reduce the amount of error and tries to come up with the best estimate, so the final output won't reflect the errors in the vision data. Perhaps something very similar is happening to the fully connected network. Even though the hidden nodes activations are sometimes mistaken, the connections between the hidden and the output tries to rid the errors so that the final output is good.

Final thoughts

As an after thought, it would have been interesting to implement a recurrent model for this lab. Since the data set is ordered - it goes from 0 1 2 ... 9 - it is actually possible to make a prediction without even looking at the inputs. If we the humans are to do this, we can just repeatedly output 0 to 9 without even looking at the input. So by adding a recurrent network we can produce the memory needed to figure this pattern out. And the network might be able to produce perfect outputs with very few connections (maybe like 10 connections total?).


Appendix

Histdiff Program

Link to our source code

This program takes in the digits.data file and returns a digits.cf file for our best network

More on Neural Networks

this man knows a lot about neural networks (click on image to hear his lecture)
[professor schwarzenegger]

Tables

Effectiveness of Different Neural Network Topologies
fullfull %guessguess %highlowhighlow %bestbest %
trial 120887%18477%18075%22293%
trial 222393%18979%19079%23498%
trial 323096%19280%15163%22192%
trial 423096%17372%20184%21991%
trial 522393%17573%18878%22895%
average222.893%17576%18276%224.894%
table 1: digits recognized out of 240, 25000 training sweeps, .5 recognition threshold


Effectiveness of Different Neural Network Topologies
fullfull %guessguess %highlowhighlow %bestbest %
trial 123899%19581%22895%240100%
trial 223799%20585%21790%240100%
trial 3240100%20786%21188%23999%
trial 4240100%20686%21288%240100%
trial 523999%21790%19280%240100%
average238.8100%20686%21288%239.8100%
table 2: digits recognized out of 240, 200000 training sweeps, .5 recognition threshold

Neural Net Graphs

[fully connected neural network archetecture]
figure 1: the "full" network. this network is fully connected.
[our guessed neural network archetecture]
figure 2: the "guess" network. this network was our educated guess at a network.
[a three eyed fish]
figure 2.1: a three eyed fish
[our highlow neural network archetecture]
figure 3: the "highlow" network. uses the most and least frequent grid spots to differentiate the digits.
[our best neural network archetecture]
figure 4: the "best" network. note the 5 fully connected hidden nodes on the right.

Cluster Analysis

(at 25,000 sweeps)

Fully Connected


Resulting Tree = 
          |-> 0l
        |-|  _|-> 0d
        | | | |-> 0q
        | |-|  _|-> 0s
        |   | | |-> 0v
        |   |-| |-> 0e
        |     | |        _|-> 0g
        |     | |     |-| |-> 0j
     |--|     |-|   |-| |_|-> 0h
     |  |       |   | |   |-> 0k
     |  |       | |-| |-> 0m
     |  |       |-| |_|-> 0f
     |  |         |   |-> 0o
 |---|  |         |_|-> 0n
 |   |  |           |-> 0w
 |   |  |-> 0x
 |   |   _|-> 8n
 |   |  | |-> 8v
 |   |--|  _|-> 8q
 |      | | |-> 8s
 |      |-| |-> 8d
 |        |-| |-> 8o
 |          | |      _|-> 8h
 |          |-|   |-| |-> 8i
 |            | |-| |-> 8j
 |            |-| |-> 8k
 |              | |-> 8l
 |              |-|  _|-> 8g
-|                |-| |-> 8t
 |                  |-> 8x
 |       _|-> 0p
 |   |--| |-> 0t
 |   |  |_|-> 0i
 |   |    |-> 0u
 |   |      __|--> 5r
 |   |     |  |__|-> 5c
 |   |     |     |-> 5t
 |   |  |--|   __|-> 5e
 |   |  |  |  |  |-> 5s
 |   |  |  |--|  |--> 5d
 |   |  |     |  |      __|-> 5f
 |   |  |     |--|  |--|  |-> 5k
 |   |  |        |  |  |--> 5u
 |   |  |        |  |        _|-> 5n
 |---|  |        |--|     |-| |-> 5o
     |  |           |  |--| |-> 5v
     |  |           |  |  |_|-> 5h
     |  |           |--|    |-> 5w
     |  |              |  |-> 5q
     |  |              |--| |-> 5j
     |  |                 |-| |-> 5m
     |  |                   |-|  _|-> 5i
     |  |                     |-| |-> 5l
     |  |                       |_|-> 5p
     |  |                         |_|-> 5g
     |  |                           |-> 5x
     |  |            _|-> 4a
     |--|        |--| |-> 4b
        |     |--|  |-> 4r
        |     |  |--> 2v
        |     |      __|--> 6c
        |  |--|     |  |__|-> 6r
        |  |  |  |--|     |-> 6t
        |  |  |  |  |__|-> 8r
        |  |  |  |     |-> 8u
        |  |  |--|      _|-> 8e
        |  |     |  |--| |-> 8f
        |  |     |  |  |-> 8m
        |  |     |--|   _|-> 0c
        |  |        |  | |_|-> 8p
        |  |        |--|   |_|-> 8c
        |  |           |     |-> 0r
        |  |           |-> 8w
        |  |      _|-> 4f
        |  |     | |_|-> 4h
        |  |  |--|   |-> 4p
        |  |  |  | |--> 4e
        |  |  |  |-|     _|-> 4n
        |--|  |    |  |-| |-> 4o
           |  |    |--| |-> 4q
           |  |       |-> 4v
           |  |      __|--> 5a
           |  |     |  |--> 6a
           |  |  |--|  |--> 4s
           |  |  |  |--|  |--> 3r
           |  |  |     |  |   __|-> 5b
           |  |  |     |--|  |  |_|-> 3a
           |  |  |        |  |    |-> 3c
           |  |  |        |--|  |-> 1f
           |  |  |           |  | |-> 1w
           |  |  |           |--| | |-> 1a
           |  |  |              | | |   |-> 1q
           |  |  |              |-| |   |     |-> 1s
           |  |  |                | |   |   |-| |-> 1n
           |--|  |                | |   |   | | |      _|-> 1g
              |  |                | |   |   | |-|   |-| |-> 1j
              |  |                |-| |-|   |   | |-| |-> 1k
              |  |                  | | | |-|   |-| |-> 1o
              |  |                  | | | | |     |  _|-> 1p
              |  |                  | | | | |     |-| |-> 1r
              |  |                  | | | | |       |-> 1t
              |  |                  | | | | | |-> 1b
              |  |                  | | |-| |-| |-> 1c
              |  |                  |-|   |   |-|    _|-> 1h
              |  |                    |   |     | |-| |-> 1l
              |  |                    |   |     |-| |-> 1m
              |  |                    |   |       |-> 1u
              |  |                    |   |_|-> 1d
              |  |                    |     |-> 1v
              |  |                    |_|-> 1e
              |  |                      |_|-> 1i
              |  |                        |-> 1x
              |  |        |--> 4c
              |--|     |--|  |--> 0a
                 |     |  |  |   __|-> 8a
                 |     |  |--|  |  |_|-> 4j
                 |     |     |  |    |-> 4m
                 |     |     |--|     |-> 8b
                 |     |        |  |--|  _|-> 4i
                 |     |        |  |  |-| |-> 4t
                 |     |        |--|    |_|-> 4d
                 |     |           |      |-> 4w
                 |     |           |__|-> 4u
                 |     |              |_|-> 4l
                 |     |                |_|-> 4k
                 |  |--|                  |_|-> 4g
                 |  |  |                    |-> 4x
                 |  |  |      __|--> 0b
                 |  |  |     |  |--> 6b
                 |  |  |     |      _|-> 6l
                 |  |  |     |     | |_|-> 6i
                 |  |  |  |--|  |--|   |_|-> 6m
                 |  |  |  |  |  |  |     |-> 6o
                 |  |  |  |  |  |  |_|-> 6d
                 |  |  |  |  |--|    |_|-> 6p
                 |  |  |  |     |      |-> 6s
                 |  |  |  |     |  |--> 6g
                 |  |  |--|     |--|   _|-> 6h
                 |  |     |        |--| |-> 6n
                 |  |     |           |  _|-> 6q
                 |  |     |           |-| |-> 6v
                 |  |     |             |-> 6w
                 |  |     |   __|-> 6e
                 |--|     |--|  |-> 6u
                    |        |__|-> 6k
                    |           |_|-> 6j
                    |             |_|-> 6f
                    |               |-> 6x
                    |      _|-> 3e
                    |     | |-> 3i
                    |  |--|  _|-> 3g
                    |  |  | | |_|-> 3f
                    |  |  |-|   |-> 3t
                    |  |    | |-> 3j
                    |  |    | |    _|-> 3l
                    |  |    |-| |-| |-> 3p
                    |  |      | | |_|-> 3m
                    |  |      |-|   |-> 3u
                    |  |        |_|-> 3h
                    |  |          |-> 3x
                    |--|      __|--> 3d
                       |  |--|  |--> 2j
                       |  |  |--> 2p
                       |  |   __|-> 2o
                       |  |  |  |-> 2r
                       |  |  |           |-> 2g
                       |  |  |        |--| |-> 7c
                       |  |  |        |  |-|  _|-> 2a
                       |  |  |        |    | | |-> 2i
                       |  |  |        |    |-| |-> 2k
                       |--|  |        |      |-|  _|-> 2h
                          |  |     |--|        | | |_|-> 9a
                          |  |     |  |        |-|   |-> 9j
                          |  |     |  |          |_|-> 2e
                          |  |     |  |            |_|-> 2l
                          |  |     |  |              |-> 2m
                          |  |     |  |  |-> 2b
                          |  |     |  |--|  _|-> 2c
                          |  |  |--|     |-| |-> 2t
                          |--|  |  |       |_|-> 2d
                             |  |  |         |-> 2x
                             |  |  |   __|-> 2n
                             |  |  |  |  |_|-> 2q
                             |  |  |  |    |_|-> 3b
                             |  |  |--|      |_|-> 2s
                             |  |     |        |-> 2w
                             |  |     |  |--> 2u
                             |  |     |--|  |-> 7e
                             |  |        |--| |-> 7a
                             |  |           |-| |-> 7g
                             |  |             |-|  _|-> 2f
                             |  |               | | |-> 7f
                             |  |               |-| |-> 7b
                             |  |                 |-| |-> 7t
                             |  |                   | |      _|-> 7h
                             |  |                   |-|   |-| |-> 7i
                             |--|                     | |-| |-> 7j
                                |                     | | |  _|-> 7k
                                |                     |-| |-| |-> 7m
                                |                       |   |-> 7o
                                |                       |_|-> 7p
                                |                         |_|-> 7u
                                |                           |_|-> 7r
                                |                             |-> 7x
                                |             _|-> 3n
                                |          |-| |-> 3q
                                |        |-| |-> 3s
                                |     |--| |-> 3v
                                |     |  |-> 3w
                                |  |--|        _|-> 7n
                                |  |  |     |-| |-> 7q
                                |  |  |  |--| |-> 7s
                                |  |  |--|  |-> 7v
                                |  |     |__|-> 3k
                                |--|        |_|-> 7l
                                   |          |_|-> 3o
                                   |            |_|-> 7d
                                   |              |-> 7w
                                   |  |--> 9b
                                   |--|  |--> 9r
                                      |--|  |-> 9l
                                         |  |   |-> 9m
                                         |--| |-| |-> 9h
                                            | | |-|  _|-> 9e
                                            | |   |-| |-> 9k
                                            |-|     |-> 9t
                                              | |-> 9o
                                              | |   |-> 9g
                                              |-| |-|    _|-> 9n
                                                | | | |-| |-> 9q
                                                | | |-| |-> 9s
                                                |-|   |-> 9v
                                                  | |-> 9u
                                                  | |    _|-> 9d
                                                  |-|   | |_|-> 9f
                                                    | |-|   |-> 9i
                                                    |-| |_|-> 9c
                                                      |   |-> 9p
                                                      |_|-> 9w
                                                        |-> 9x

Guess Tree


Resulting Tree = 
 |--> 9a
 |      _|-> 1e
 |  |--| |-> 1f
 |  |  |_|-> 1d
-|  |    |-> 1v
 |  |  |--> 1b
 |  |  |        |-> 0e
 |  |  |     |--|  _|-> 0p
 |--|  |     |  |-| |-> 0t
    |  |     |    |_|-> 0i
    |  |     |      |-> 0u
    |  |     |     _|-> 0k
    |  |  |--|    | |-> 0m
    |  |  |  |  |-|  _|-> 0h
    |--|  |  |  | | | |_|-> 0g
       |  |  |  | |-|   |-> 0j
       |  |  |  |   |_|-> 0s
       |  |  |--|     |-> 0v
       |  |     |    _|-> 0f
       |  |     |   | |-> 0o
       |  |     | |-|  _|-> 0d
       |  |     | | |-| |-> 0q
       |  |     |-|   |_|-> 0n
       |--|       |     |-> 0w
          |       |_|-> 0l
          |         |-> 0x
          |   __|-> 7a
          |  |  |-> 2r
          |  |   __|-> 2v
          |  |  |  |_|-> 2q
          |  |  |    |_|-> 2s
          |  |  |      |-> 2w
          |  |  |         _|-> 1a
          |  |  |        | |-> 1i
          |--|  |        |        _|-> 1h
             |  |        |     |-| |-> 1l
             |  |        |     | |-> 1m
             |  |     |--|   |-|    _|-> 1g
             |  |     |  |   | | |-| |-> 1j
             |  |     |  | |-| |-| |-> 1k
             |  |     |  | | |   |-> 1o
             |  |  |--|  |-| |_|-> 1q
             |--|  |  |    |   |-> 1s
                |  |  |    |  _|-> 1p
                |  |  |    |-| |-> 1r
                |  |  |      |-> 1t
                |  |  |__|-> 1c
                |  |     |-> 1u
                |  |         _|-> 6f
                |  |     |--| |-> 6r
                |  |     |  |_|-> 6h
                |  |     |    |-> 6w
                |  |     |           _|-> 4a
                |  |     |        |-| |-> 4b
                |  |     |     |--| |-> 4d
                |  |     |     |  |-> 4r
                |--|  |--|  |--|  |-> 0b
                   |  |  |  |  |  | |-> 6n
                   |  |  |  |  |--| |    _|-> 2b
                   |  |  |  |     | |   | |_|-> 4f
                   |  |  |  |     |-| |-|   |-> 4h
                   |  |  |  |       | | | |-> 4s
                   |  |  |--|       | | |-| |-> 4p
                   |  |     |       | |   |-| |-> 4e
                   |  |     |       |-|     |-|    _|-> 4n
                   |  |     |         |       | |-| |-> 4o
                   |  |     |         |       |-| |-> 4q
                   |  |     |         |         |-> 4v
                   |  |     |         |-> 4w
                   |  |     |__|-> 7b
                   |  |        |_|-> 1n
                   |--|          |-> 1x
                      |  |--> 9j
                      |  |      __|--> 0a
                      |  |  |--|  |--> 0r
                      |  |  |  |__|-> 9r
                      |  |  |     |-> 6t
                      |  |  |         _|-> 2j
                      |  |  |      |-| |-> 2o
                      |  |  |      | |-> 2u
                      |  |  |    |-|    _|-> 7r
                      |  |  |    | | |-| |-> 7t
                      |  |  |    | | | |-> 7u
                      |  |  |    | |-|    _|-> 2d
                      |  |  |    |   | |-| |-> 2h
                      |  |  |    |   |-| |-> 2i
                      |--|  |    |     |-> 2x
                         |  |    |        _|-> 4c
                         |  |    |     |-| |-> 4g
                         |  |    |     | |_|-> 3a
                         |  |    |     |   |-> 5r
                         |  |    |   |-|  _|-> 0c
                         |  |    |   | | | |-> 5d
                         |  |    |   | |-|   |-> 2a
                         |  |    |   |   | |-|  _|-> 8e
                         |  |    |   |   |-| |-| |-> 8f
                         |  |  |-|   |     |   |-> 8m
                         |  |  | | |-|     |_|-> 8a
                         |  |  | | | |       |-> 5s
                         |  |  | | | |    _|-> 8c
                         |--|  | | | |   | |_|-> 8q
                            |  | | | | |-|   |-> 8s
                            |  | | | | | |_|-> 8d
                            |  | | | | |   |_|-> 8n
                            |  | | | |-|     |-> 8v
                            |  | | |   | |-> 5u
                            |  | | |   | |      _|-> 5f
                            |  | | |   |-|   |-| |_|-> 5e
                            |  | | |     | |-| |   |-> 5m
                            |  | | |     | | | |-> 5q
                            |  | | |     |-| |-> 5t
                            |  | | |       | |-> 5p
                            |  | |-|       |-| |-> 5c
                            |  |   |         |-| |-> 5j
                            |  |   |           | |      _|-> 5h
                            |  |   |           |-|   |-| |-> 5n
                            |  |   |             | |-| |-> 5o
                            |  |   |             |-| |-> 5v
                            |  |   |               |  _|-> 5i
                            |  |   |               |-| |-> 5l
                            |  |   |                 |_|-> 5k
                            |  |   |                   |_|-> 5w
                            |  |   |                     |_|-> 5g
                            |  |   |                       |-> 5x
                            |  |   |    _|-> 8o
                            |  |   | |-| |-> 8p
                            |--|   | | |-> 8w
                               |   |-| |-> 8l
                               |     | |      _|-> 8h
                               |     |-|   |-| |-> 8i
                               |       | |-| |-> 8j
                               |       |-| |-> 8k
                               |         |  _|-> 8g
                               |         |-| |-> 8t
                               |           |-> 8x
                               |  _|-> 5a
                               | | |_|-> 6a
                               | |   |-> 1w
                               | |       |-> 3r
                               | |       |   |-> 9b
                               | |     |-| |-| |-> 7d
                               | |     | | | |-|    _|-> 7n
                               | |     | | |   | |-| |-> 7q
                               | |     | |-|   |-| |-> 7s
                               | |     |   |     |-> 7v
                               | |     |   | |-> 5b
                               | |     |   |-|  _|-> 7e
                               | |     |     | | |-> 7l
                               | |   |-|     |-|  _|-> 7c
                               | |   | |       |-| |-> 3e
                               | |   | |         |  _|-> 3i
                               | |   | |         |-| |-> 3o
                               | |   | |           |-> 7w
                               | |   | |    _|-> 3g
                               | |   | | |-| |-> 3j
                               | |   | | | |_|-> 3k
                               | |   | |-|   |-> 3m
                               |-|   |   | |-> 3p
                                 |   |   |-|  _|-> 3l
                                 |   |     |-| |-> 3u
                                 |   |       |_|-> 3h
                                 |   |         |-> 3x
                                 | |-|     |-> 7g
                                 | | |     |   |-> 2g
                                 | | |     | |-| |-> 2n
                                 | | |   |-| | |-| |-> 2f
                                 | | |   | | |   | |        _|-> 7h
                                 | | |   | | |   |-|     |-| |-> 7i
                                 | | |   | | |     |   |-| |-> 7j
                                 | | |   | |-|     | |-| |-> 7k
                                 | | |   |   |     |-| |-> 7m
                                 | | | |-|   |       |-> 7o
                                 | | | | |   |  _|-> 2k
                                 | | | | |   | | |_|-> 7f
                                 | | | | |   |-|   |_|-> 2l
                                 | | | | |     |     |-> 2m
                                 | | |-| |     |-> 2p
                                 | |   | |  _|-> 2c
                                 |-|   | |-| |-> 2e
                                   |   |   |-> 2t
                                   |   |_|-> 7p
                                   |     |-> 7x
                                   |    _|-> 6q
                                   | |-| |-> 6v
                                   | | |  _|-> 6g
                                   | | |-| |-> 6l
                                   | |   |-> 6x
                                   | |       |-> 6j
                                   | |     |-|  _|-> 6m
                                   | |     | |-| |-> 6o
                                   | |     |   |_|-> 6p
                                   | |   |-|     |-> 6s
                                   | |   | |  _|-> 3b
                                   | |   | |-| |_|-> 6b
                                   |-| |-|   |   |-> 8b
                                     | | |   |-> 6u
                                     | | | |-> 6i
                                     | | |-| |-> 6d
                                     | |   |-|  _|-> 6c
                                     | |     |-| |-> 6e
                                     | |       |_|-> 8r
                                     | |         |-> 8u
                                     | |     |-> 3c
                                     | |     |   |-> 9m
                                     | |     | |-| |-> 9d
                                     |-|   |-| | | |      _|-> 3n
                                       |   | | | |-|   |-| |-> 9n
                                       |   | | |   | |-| |-> 3q
                                       |   | |-|   | | |  _|-> 9q
                                       |   |   |   |-| |-| |-> 3s
                                       | |-|   |     |   |-> 9s
                                       | | |   |     |_|-> 3v
                                       | | |   |       |-> 9v
                                       | | |   |-> 3w
                                       | | |  _|-> 3f
                                       | | |-| |-> 9o
                                       | |   |_|-> 3d
                                       |-|     |-> 9w
                                         |    _|-> 4j
                                         | |-| |-> 4m
                                         | | | |-> 4t
                                         | | |-|  _|-> 4k
                                         | |   |-| |-> 4l
                                         | |     |-> 4u
                                         |-|   |-> 9i
                                           | |-|  _|-> 9c
                                           | | |-| |-> 9p
                                           | |   |  _|-> 9e
                                           |-|   |-| |-> 9k
                                             |     |-> 9t
                                             | |-> 4x
                                             |-|  _|-> 3t
                                               | | |-> 9u
                                               |-| |-> 4i
                                                 |-|  _|-> 9f
                                                   | | |_|-> 6k
                                                   |-|   |_|-> 9h
                                                     |     |-> 9l
                                                     |_|-> 9g
                                                       |-> 9x

HighLow Tree


Resulting Tree = 
 |---> 4c
 |       |---> 4t
 |       |    __|--> 4i
 |   |---|   |  |--> 6t
 |   |   |   |  |--> 7c
 |   |   |---|  |     |--> 4e
 |   |       |  |  |--|  |-> 4r
 |   |       |--|  |  |--|  _|-> 4a
 |   |          |  |     |-| |-> 4b
 |   |          |  |       |-> 4s
 |   |          |  |     |--> 4u
 |   |          |--|  |--|   __|--> 4g
 |   |             |  |  |  |  |__|-> 4k
 |   |             |  |  |--|     |-> 4l
 |   |             |  |     |   _|-> 4d
 |   |             |  |     |--| |-> 4p
-|   |             |--|        |  _|-> 4f
 |   |                |        |-| |-> 4h
 |   |                |          |-> 4w
 |   |                |       _|-> 4n
 |   |                |    |-| |-> 4o
 |   |                |  |-| |-> 4q
 |   |                |--| |-> 4v
 |   |                   |-> 4x
 |   |           _|-> 1e
 |   |       |--| |-> 1f
 |   |       |  |_|-> 1d
 |   |       |    |-> 1v
 |   |   |---|   __|--> 5a
 |   |   |   |  |  |__|-> 1b
 |   |   |   |  |     |-> 1w
 |   |   |   |--|   __|-> 1c
 |---|   |      |  |  |-> 1u
     |   |      |  |         _|-> 1h
     |   |      |--|     |--| |-> 1l
     |   |         |  |--|  |-> 1m
     |   |         |  |  |__|-> 1q
     |   |         |  |     |-> 6u
     |   |         |--|   __|--> 1a
     |   |            |  |  |--> 1i
     |   |            |  |      _|-> 1p
     |   |            |--|  |--| |-> 1r
     |   |               |  |  |-> 1t
     |   |               |  |    |-> 1n
     |   |               |--|    |      _|-> 1g
     |   |                  |  |-|   |-| |-> 1j
     |   |                  |  | | |-| |-> 1k
     |   |                  |--| |-| |-> 1o
     |   |                     |   |-> 1s
     |   |                     |-> 1x
     |   |       |--> 6f
     |   |       |  |--> 6r
     |   |       |  |           _|-> 6g
     |   |   |---|  |        |-| |-> 6h
     |   |   |   |  |     |--| |-> 6l
     |   |   |   |  |     |  | |-> 6k
     |   |   |   |--|     |  |-|  _|-> 6e
     |   |   |      |     |    |-| |-> 6i
     |   |   |      |  |--|      |_|-> 6m
     |   |   |      |  |  |        |-> 6o
     |   |   |      |  |  |   __|-> 6n
     |---|   |      |  |  |  |  |_|-> 6p
         |   |      |--|  |--|    |-> 6s
         |   |         |     |__|-> 6j
         |   |         |        |_|-> 6d
         |   |         |          |-> 6w
         |   |         |--> 6x
         |   |         |--> 5k
         |   |         |  |--> 5f
         |   |      |--|  |    |-> 5e
         |   |      |  |  |  |-| |-> 5m
         |   |      |  |--|  | |-| |-> 5s
         |   |      |     |  |   |-|    _|-> 5p
         |   |      |     |  |     | |-| |-> 5q
         |   |      |     |  |     |-| |_|-> 5d
         |   |      |     |--|       |   |-> 5r
         |   |      |        |       |-> 5t
         |   |      |        |   |-> 5h
         |   |      |        | |-|  _|-> 5n
         |   |      |        | | |-| |-> 5o
         |   |      |        |-|   |-> 5v
         |   |      |          |  _|-> 5i
         |   |      |          |-| |-> 5l
         |   |      |            |_|-> 5w
         |   |      |              |_|-> 5j
         |   |      |                |_|-> 5g
         |   |      |                  |-> 5x
         |   |      |         __|-> 3b
         |   |   |--|     |--|  |-> 5b
         |   |   |  |     |  |   __|--> 3a
         |   |   |  |     |  |--|  |--> 4j
         |   |   |  |     |     |--> 4m
         |   |   |  |  |--|      _|-> 0a
         |   |   |  |  |  |  |--| |-> 0c
         |   |   |  |  |  |  |  |-> 0r
         |---|   |  |  |  |--|      __|-> 8a
             |   |  |  |     |  |--|  |-> 8c
             |   |  |  |     |--|  |--> 8p
             |   |  |  |        |__|-> 8r
             |   |  |  |           |-> 8u
             |   |  |  |        |--> 0b
             |   |  |  |     |--|   _|-> 0i
             |   |  |  |     |  |  | |-> 0u
             |   |  |  |     |  |--|  _|-> 0e
             |   |  |  |     |     | | |-> 0m
             |   |  |  |     |     |-|  _|-> 6q
             |   |  |  |     |       |-| |-> 6v
             |   |  |--|     |         |_|-> 2s
             |   |     |     |           |-> 2w
             |   |     |  |--|       _|-> 0f
             |   |     |  |  |    |-| |-> 0o
             |   |     |  |  |    | |_|-> 0s
             |   |     |  |  |  |-|   |-> 0v
             |   |     |  |  |  | | |-> 0h
             |   |     |  |  |  | |-|  _|-> 0g
             |   |     |  |  |  |   |-| |-> 0j
             |   |     |  |  |--|     |-> 2v
             |   |     |  |     |   |-> 0k
             |   |     |  |     | |-| |-> 8d
             |   |     |  |     | | |-|  _|-> 8n
             |   |     |  |     | |   |-| |-> 8v
             |   |     |  |     |-|     |_|-> 0n
             |   |     |  |       |       |-> 0w
             |   |     |--|       |  _|-> 0d
             |   |        |       | | |-> 0q
             |   |        |       |-|  _|-> 0p
             |   |        |         |-| |-> 0t
             |   |        |           |_|-> 0l
             |   |        |             |-> 0x
             |   |        |         __|-> 8e
             |   |        |     |--|  |-> 8f
             |   |        |     |  |__|-> 8m
             |   |        |  |--|     |-> 8o
             |   |        |  |  |  |--> 6a
             |   |        |  |  |--|   _|-> 6b
             |   |        |  |     |--| |-> 8b
             |---|        |  |        |  _|-> 8q
                 |        |  |        |-| |-> 8s
                 |        |--|          |-> 8w
                 |           |         _|-> 8h
                 |           |      |-| |-> 8i
                 |           |    |-| |-> 8j
                 |           |  |-| |-> 8k
                 |           |--| |-> 8l
                 |              |  _|-> 8g
                 |              |-| |-> 8t
                 |                |-> 8x
                 |         _|-> 3l
                 |     |--| |_|-> 3i
                 |     |  |   |-> 3t
                 |     |  |-> 3u
                 |     |   __|--> 2d
                 |  |--|  |  |--> 2x
                 |  |  |  |   __|--> 2a
                 |  |  |  |  |  |--> 2b
                 |  |  |--|  |   __|-> 3f
                 |  |     |  |  |  |_|-> 9b
                 |  |     |  |  |    |_|-> 3e
                 |  |     |  |  |      |-> 3o
                 |  |     |--|  |           _|-> 3m
                 |  |        |  |        |-| |-> 2n
                 |  |        |  |        | |_|-> 2f
                 |  |        |  |     |--|   |_|-> 2r
                 |  |        |  |     |  |     |_|-> 3c
                 |  |        |  |     |  |       |-> 3r
                 |  |        |  |     |  |-> 9u
                 |  |        |--|  |--|     _|-> 2e
                 |  |           |  |  |    | |_|-> 2g
                 |  |           |  |  |  |-|   |_|-> 2l
                 |  |           |  |  |  | |     |-> 2m
                 |  |           |  |  |--| |-> 2p
                 |  |           |  |     |_|-> 3g
                 |  |           |  |       |_|-> 3j
                 |  |           |  |         |-> 3x
                 |  |           |  |         __|-> 2c
                 |  |           |  |     |--|  |-> 2t
                 |  |           |--|     |  |__|-> 5c
                 |  |              |     |     |_|-> 9a
                 |  |              |     |       |-> 5u
                 |  |              |  |--|      _|-> 2i
                 |  |              |  |  |  |--| |-> 2j
                 |  |              |  |  |  |  |_|-> 2h
                 |  |              |  |  |--|    |-> 2o
                 |  |              |  |     |  |-> 7e
                 |  |              |  |     |--|    _|-> 7r
                 |--|              |  |        | |-| |-> 7t
                    |              |--|        |-| |_|-> 7g
                    |                 |          |   |-> 7u
                    |                 |          |-> 7w
                    |                 |   __|-> 7a
                    |                 |  |  |_|-> 7b
                    |                 |  |    |-> 2k
                    |                 |--|   __|-> 3h
                    |                    |  |  |_|-> 3d
                    |                    |  |    |-> 3p
                    |                    |  |     _|-> 7d
                    |                    |--|  |-| |-> 7l
                    |                       |  | |-> 2u
                    |                       |  |   |-> 2q
                    |                       |--| |-| |-> 3k
                    |                          | | |-|    _|-> 7n
                    |                          | |   | |-| |-> 7q
                    |                          | |   |-| |-> 7s
                    |                          |-|     |-> 7v
                    |                            | |-> 7p
                    |                            | |          _|-> 7h
                    |                            | |       |-| |-> 7i
                    |                            |-|     |-| |-> 7j
                    |                              |   |-| |-> 7k
                    |                              | |-| |-> 7m
                    |                              |-| |-> 7o
                    |                                |_|-> 7f
                    |                                  |-> 7x
                    |  |--> 9l
                    |  |     |-> 9j
                    |  |  |--|  _|-> 9f
                    |  |  |  |-| |-> 9g
                    |  |  |    | |-> 9i
                    |--|  |    |-| |-> 9m
                       |  |      |-| |-> 9d
                       |  |        |-|  _|-> 9e
                       |  |          |-| |-> 9k
                       |  |            |-> 9t
                       |--|    |-> 9r
                          |    |      _|-> 3n
                          |  |-|   |-| |-> 9n
                          |  | | |-| |-> 3q
                          |  | | | |  _|-> 9q
                          |  | |-| |-| |-> 3s
                          |  |   |   |-> 9s
                          |--|   |_|-> 3v
                             |     |-> 9v
                             |   |-> 6c
                             | |-|  _|-> 9c
                             | | |-| |-> 9p
                             |-|   |_|-> 3w
                               |     |_|-> 9o
                               |       |-> 9w
                               |_|-> 9h
                                 |-> 9x

Best Tree

Resulting Tree = 
 |---> 4c
 |       |---> 2v
 |       |    __|-> 8n
 |   |---|   |  |-> 8v
 |   |   |   |  |--> 8p
 |   |   |---|  |      |-> 8m
 |   |       |  |    |-|  _|-> 8e
 |   |       |  |    | |-| |-> 8f
 |   |       |--|  |-|   |_|-> 8d
 |   |          |  | |     |_|-> 8c
 |   |          |  | |       |-> 8o
 |   |          |  | |_|-> 8r
 |   |          |  |   |-> 8u
 |   |          |--|    _|-> 8q
-|   |             | |-| |-> 8s
 |   |             | | |-> 8w
 |   |             | |        _|-> 8h
 |   |             |-|     |-| |-> 8i
 |   |               |   |-| |-> 8j
 |   |               | |-| |-> 8k
 |   |               |-| |-> 8l
 |   |                 |  _|-> 8g
 |   |                 |-| |-> 8t
 |   |                   |-> 8x
 |   |          |-> 0i
 |   |       |--|  _|-> 0e
 |   |       |  |-| |-> 0m
 |   |       |    |-> 0u
 |---|   |---|   _|-> 0s
     |   |   |  | |-> 0v
     |   |   |  |   |-> 0c
     |   |   |  |   |  _|-> 0h
     |   |   |--| |-| | |-> 0r
     |   |      | | | |    _|-> 0g
     |   |      | | |-| |-| |-> 0j
     |   |      | |   | | |_|-> 0k
     |   |      |-|   |-|   |_|-> 0f
     |   |        |     |     |-> 0o
     |   |        |     |_|-> 0p
     |   |        |       |-> 0t
     |   |        | |-> 0l
     |   |        |-|    _|-> 0d
     |   |          | |-| |-> 0q
     |   |          |-| |_|-> 0n
     |   |            |   |-> 0w
     |   |            |-> 0x
     |   |              _|-> 5c
     |   |          |--| |-> 5r
     |   |       |--|  |-> 5t
     |   |       |  |--> 5u
     |   |       |      __|-> 5d
     |   |   |---|  |--|  |_|-> 5f
     |---|   |   |  |  |    |-> 5m
         |   |   |  |  |--> 5p
         |   |   |--|   __|-> 5e
         |   |      |  |  |_|-> 5q
         |   |      |  |    |_|-> 5k
         |   |      |--|      |-> 5s
         |   |         |  |-> 5j
         |   |         |  |  _|-> 5i
         |   |         |--| | |-> 5l
         |   |            | |      _|-> 5n
         |   |            |-|   |-| |-> 5o
         |   |              | |-| |-> 5v
         |   |              |-| |_|-> 5h
         |   |                |   |-> 5w
         |   |                |_|-> 5g
         |   |                  |-> 5x
         |   |          __|-> 4a
         |   |         |  |-> 4b
         |   |         |      __|-> 4j
         |   |      |--|  |--|  |-> 4m
         |   |      |  |  |  |__|--> 4r
         |   |      |  |  |     |--> 4t
         |   |      |  |--|      __|-> 4g
         |   |      |     |     |  |_|-> 4d
         |---|      |     |  |--|    |-> 4h
             |      |     |  |  |__|-> 4k
             |      |     |--|     |-> 4l
             |      |        |  |-> 4u
             |   |--|        |--| |-> 4e
             |   |  |           |-| |-> 4s
             |   |  |             |-| |-> 4f
             |   |  |               | |   |-> 4p
             |   |  |               |-| |-|    _|-> 4n
             |   |  |                 | | | |-| |-> 4o
             |   |  |                 |-| |-| |-> 4q
             |   |  |                   |   |-> 4v
             |   |  |                   |-> 4w
             |   |  |__|-> 4i
             |   |     |-> 4x
             |   |      |--> 6a
             |   |      |   __|--> 6f
             |   |  |---|  |  |--> 6j
             |   |  |   |  |   __|-> 6m
             |   |  |   |--|  |  |-> 6o
             |   |  |      |  |    |-> 6k
             |   |  |      |--|  |-|  _|-> 6p
             |   |  |         |  | |-| |-> 6s
             |   |  |         |  |   |-> 6u
             |   |  |         |  |   |-> 6c
             |---|  |         |--| |-| |-> 6i
                 |  |            | | |-|  _|-> 6e
                 |  |            | |   | | |_|-> 6q
                 |  |            | |   |-|   |-> 6v
                 |  |            |-|     |  _|-> 6d
                 |  |              |     |-| |-> 6n
                 |  |              |       |-> 6w
                 |  |              |_|-> 6t
                 |  |                |_|-> 6l
                 |  |                  |_|-> 6h
                 |  |                    |_|-> 6r
                 |  |                      |_|-> 6g
                 |  |                        |-> 6x
                 |  |       |--> 1c
                 |  |       |     |-> 1i
                 |  |       |  |--|  _|-> 1a
                 |  |   |---|  |  | | |-> 1q
                 |  |   |   |  |  |-|  _|-> 1e
                 |  |   |   |  |    |-| |-> 1f
                 |  |   |   |--|      |_|-> 1d
                 |  |   |      |        |-> 1v
                 |  |   |      |  |-> 1w
                 |--|   |      |--| |-> 1u
                    |   |         | | |-> 1b
                    |   |         |-| |   |-> 1n
                    |   |           | |   |        _|-> 1g
                    |   |           | |   |     |-| |-> 1j
                    |   |           |-| |-|   |-| |-> 1k
                    |   |             | | | |-| |-> 1o
                    |   |             | | | | |-> 1s
                    |   |             | | |-|    _|-> 1h
                    |   |             |-|   | |-| |-> 1l
                    |   |               |   |-| |-> 1m
                    |   |               |     |  _|-> 1p
                    |   |               |     |-| |-> 1r
                    |   |               |       |-> 1t
                    |   |               |-> 1x
                    |   |         |--> 2u
                    |   |         |      __|--> 2g
                    |   |         |     |  |--> 2i
                    |   |         |  |--|      __|--> 2a
                    |   |      |--|  |  |  |--|  |--> 2d
                    |   |      |  |  |  |--|  |--> 2p
                    |   |      |  |  |     |  |-> 2k
                    |   |      |  |  |     |--| |-> 2h
                    |   |      |  |--|        |-|  _|-> 2e
                    |   |      |     |          | | |_|-> 2l
                    |   |      |     |          |-|   |-> 2m
                    |   |      |     |            |_|-> 2c
                    |   |      |     |              |-> 2t
                    |   |      |     |__|--> 2o
                    |   |      |        |__|--> 2j
                    |---|      |           |__|-> 2r
                        |      |              |-> 2x
                        |      |         _|-> 0a
                        |   |--|     |--| |-> 0b
                        |   |  |     |  |_|-> 6b
                        |   |  |     |    |_|-> 8a
                        |   |  |  |--|      |-> 8b
                        |   |  |  |  |   __|-> 2b
                        |   |  |  |  |--|  |-> 2q
                        |   |  |  |     |__|-> 2s
                        |   |  |  |        |-> 2w
                        |   |  |  |     |--> 3d
                        |   |  |  |     |      __|--> 3a
                        |   |  |  |     |     |  |--> 7c
                        |   |  |  |     |     |     |--> 5a
                        |   |  |  |     |  |--|  |--|  |--> 3o
                        |   |  |  |  |--|  |  |  |  |  |       _|-> 3n
                        |   |  |  |  |  |  |  |  |  |--|    |-| |-> 3q
                        |   |  |--|  |  |  |  |  |     |  |-| |-> 3s
                        |   |     |  |  |  |  |--|     |--| |-> 3v
                        |   |     |  |  |  |     |        |-> 3w
                        |   |     |  |  |--|     |   _|-> 3b
                        |   |     |  |     |     |--| |-> 9b
                        |   |     |  |     |        |_|-> 5b
                        |   |     |  |     |          |-> 9w
                        |   |     |  |     |  |--> 3e
                        |   |     |  |     |  |  |-> 3c
                        |   |     |  |     |--|  |        _|-> 3g
                        |   |     |  |        |  |     |-| |-> 3j
                        |   |     |  |        |  |     | |_|-> 3i
                        |   |     |--|        |--|   |-|   |_|-> 3k
                        |   |        |           |   | |     |-> 3m
                        |   |        |           | |-| |-> 3t
                        |   |        |           | | |_|-> 3f
                        |   |        |           |-|   |_|-> 3l
                        |   |        |             |     |-> 3u
                        |   |        |             |  _|-> 3p
                        |---|        |             |-| |-> 3r
                            |        |               |_|-> 3h
                            |        |                 |-> 3x
                            |        |  |--> 7b
                            |        |  |   __|-> 2f
                            |        |  |  |  |-> 2n
                            |        |--|  |       |-> 7l
                            |           |  |     |-|  _|-> 7d
                            |           |  |     | |-| |-> 7g
                            |           |  |     |   |_|-> 7f
                            |           |--|  |--|     |-> 7p
                            |              |  |  |      _|-> 7n
                            |              |  |  |   |-| |-> 7q
                            |              |  |  | |-| |-> 7s
                            |              |  |  |-| |-> 7v
                            |              |  |    |-> 7w
                            |              |  |         _|-> 7h
                            |              |--|      |-| |-> 7i
                            |                 |    |-| |-> 7j
                            |                 |    | |  _|-> 7k
                            |                 |    | |-| |-> 7m
                            |                 |  |-|   |-> 7o
                            |                 |  | |  _|-> 7e
                            |                 |  | | | |_|-> 7r
                            |                 |--| |-|   |-> 7t
                            |                    |   |_|-> 7a
                            |                    |     |-> 7u
                            |                    |-> 7x
                            |   __|--> 9a
                            |  |  |--> 9u
                            |  |     |--> 9l
                            |  |  |--|  |--> 9j
                            |  |  |  |--|   _|-> 9m
                            |--|  |     |  | |_|-> 9c
                               |  |     |--|   |-> 9p
                               |  |        | |-> 9o
                               |  |        |-|    _|-> 9n
                               |  |          | |-| |-> 9q
                               |--|          |-| |-> 9s
                                  |            |-> 9v
                                  |     _|-> 9d
                                  |    | |-> 9g
                                  |  |-|    _|-> 9f
                                  |  | | |-| |-> 9i
                                  |  | |-| |-> 9r
                                  |--|   |  _|-> 9e
                                     |   |-| |-> 9k
                                     |     |-> 9t
                                     |_|-> 9h
                                       |-> 9x