STYLE ON THE STREET (Representation & Reasoning)

Reasoning algorithms

Learning Objective: Compare several algorithms that could be used to solve a specific type of reasoning problem.

Enduring Understanding: The choice of reasoning algorithm depends on the characteristics of the input data and the types of decisions to be made.

Unpacked: For classifying dogs vs. cats, if all we have are raw images, we cannot use a decision tree because it would require an astronomical number of nodes; we must use a neural network because it can efficiently perform feature extraction. But if we already have a description of an animal in terms of features such as snout length and ear shape, then a decision tree could be used.

In general, classification problems can be solved using decision trees, nearest-neighbor algorithms, or neural networks. Prediction problems can be solved using linear regression or neural networks. Combinatorial search can make use of several different algorithms for growing the search tree. Sequential decision problems can use either Q tables or neural networks to choose the best action.

In the previous blog, we talked about perception, especially image recognition.

In my story, I want to find a pair of sunglasses, and Google Images produces an awesome result. We also mentioned why edges are important for a convolutional neural network to 'see'. This lays a solid foundation for us to move on to today’s topic.

Have you considered this possibility: Instead of uploading a photo, could I just tell the computer that I am looking for a pair of brown glasses with a V-shaped frame? Which way is better? Let us tell about the decision tree!

Decision trees and neural networks are both popular machine learning algorithms, but they differ in their structure, operation, and use cases.

1. Structure:

   • Decision Tree: Decision trees are hierarchical structures composed of nodes, where each node represents a decision based on a feature of the input data. The tree branches out based on these decisions until a prediction or decision is made at the leaf nodes.

   • Neural Network: Neural networks are interconnected networks of artificial neurons organized into layers. They typically consist of an input layer, one or more hidden layers, and an output layer. Each neuron receives inputs, performs a computation, and passes the result to the next layer.

2. Operation:

   • Decision Tree: Decision trees make decisions by splitting the data based on the values of input features. These splits are chosen to maximize the information gain or minimize impurity at each node.

   • Neural Network: Neural networks learn by adjusting the weights associated with connections between neurons during a training process. They use activation functions to introduce non-linearity into the model and learn complex patterns in the data.

3. Use Cases:

   • Decision Tree: Decision trees are often used for classification and regression tasks. They are interpretable and easy to understand, making them suitable for tasks where transparency and explainability are important.

   • Neural Network: Neural networks are powerful models capable of learning complex patterns from large amounts of data. They are widely used in applications such as image recognition, natural language processing, and speech recognition, where they often outperform other machine learning algorithms.

4. Interpretability:

   • Decision Tree: Decision trees are inherently interpretable, as the decision-making process can be visualized and understood by humans.

   • Neural Network: Neural networks are typically considered black-box models, meaning that their internal workings are not easily interpretable. While techniques such as feature visualization and model interpretation can provide insights into their behavior, neural networks are generally less interpretable than decision trees.

To use a decision tree model, I need to instruct the computer using the same features (nodes) provided by the training data. However, it is impossible for me as a user to know all the features. Even if I have extensive knowledge of sunglasses types, I may not be aware of all the relevant features. Therefore, if I am using a raw image, I need to use a neural network, although it is less interpretable than decision trees. For example, it cannot explain why a pair of sunglasses is different from a regular one.

While AI agents can reason about very complex problems, they do not think the way humans do. If I ask glasses shoppers, they can not only tell me the brand and its designer but also provide an explanation.