Tips
A small glossary of the basic words that are used in the world of ChatGPT, there are many more but little by little we will learn them
Adversary Training: A technique for improving the robustness of language models by exposing them to examples specifically designed to challenge the model and force it to learn more robust representations.
AI (Artificial Intelligence): Field of computing focused on building intelligent machines that can perform tasks that normally require human intelligence, such as speech recognition, problem solving, and language translation.
Attention Mechanism: A technique used in Transformer models to selectively focus on certain tokens in the input stream when generating the output.
Autoregression: A property of language models, where the prediction of the next token depends on previous tokens generated by the model itself, rather than just on the input.
Beam Search: Search algorithm used in PLN to find the most probable sequence of tokens generated by a language model, maintaining a bundle of the k most probable sequences at each time step.
Contextual Embeds: Word embeddings that are generated in a way that considers the context in which the words appear, such as surrounding words or sentence structure.
Coreference Resolution: The process of identifying when different expressions in a text refer to the same entity and replacing them with a single, coherent representation.
Dependency Analysis: A task in PLN that involves analyzing the grammatical structure of a sentence to identify the relationships between its components, such as subject, object, or modifier.
Implementation: The process of making a trained language model available, either by integrating it into a larger system or by providing an API for others to access.
Entities: Refers to real-world objects, such as people, organizations, locations, or products, that can be identified and extracted from the text.
Evaluation Metrics: Measures used to evaluate the performance of a language model, such as puzzlement, accuracy, F1 score, or BLEU score.
Fine Tuning: The process of adapting a pretrained language model for a specific task by training it on a smaller task-specific data set.
Fine Granularity Control: The ability of a language model to generate text with specific attributes, such as style, tone, or content, by adjusting its internal parameters.
Generation: The process of using a language model to generate new text, either by sampling from the model's predicted distribution over tokens, or by using the model as a guide for human text generation.
Generative Antagonistic Networks (GAN): A type of neural network architecture used in NLP, composed of two models: a generator that generates text, and a discriminator that evaluates the quality of the generated text and provides feedback to the generator.
GPT (Generative Pre-trained Transformer): An AI language model developed by OpenAI, trained on a large corpus of internet text to generate human-like text.
Greedy Search: Search algorithm used in PLN to find the most probable sequence of tokens generated by a language model, selecting the most probable token at each time step.
Inference: The process of using a trained language model to make predictions about new, unseen data.
Knowledge Base: A structured repository of information, such as a database or an ontology, that can be used to provide context and background information for language models.
Language Model: An AI model that has been trained to generate text based on patterns it has learned from a large corpus of text data.
Masked Language Modeling: A pretraining task in which some tokens in the input stream are masked, and the model is trained to predict these tokens, given contextual information from the surrounding tokens.
Multitask Learning: The process of training a model on multiple tasks simultaneously, in order to improve overall performance and learn shared representations between tasks.
Named Entity Recognition (NER): A task in PLN that involves identifying and classifying entities mentioned in a text into predefined categories, such as person, location, or organization.
PLN (Natural Language Processing): Subfield of AI focused on the interaction between computers and humans through natural language.
Overfitting: A common problem in machine learning where a model becomes too specialized on the training data and performs poorly on unseen data. To avoid overfitting, models are often regularized, for example, by using "dropout" or early stopping.