Author: ultroni1

The Transformer architecture has allowed us to train models for Natural Language Processing (NLP) in a more efficient way. Instead of processing each token in a sentence or sequence, attention allows a model to process tokens in parallel in various ways.

To train a model using the Transformer architecture, you need to use a large amount of text data as input. Different models have been trained, which mostly differ by the data they’ve been trained on, or by how they implement attention within their architectures. Since the models are trained on large datasets, and the models themselves are large in size, they’re often referred to as Large Language Models (LLMs).

Many LLMs are open-source and publicly available through communities like Hugging Face. Azure also offers the most commonly used LLMs as foundation models in the Azure Machine Learning model catalog. Foundation models are pretrained on large texts and can be fine-tuned for specific tasks with a relatively small dataset.

Explore the model catalog

In the Azure Machine Learning studio, you can navigate to the model catalog to explore all available foundation models. Additionally, you can import any model from the Hugging Face open-source library into the model catalog.

clients

The latest breakthrough in Natural Language Processing (NLP) is owed to the development of the Transformer architecture.

Transformers were introduced in the Attention is all you need paper by Vaswani, et al. from 2017. The Transformer architecture provides an alternative to the Recurrent Neural Networks (RNNS) to do NLP. Whereas RNNs are compute-intensive since they process words sequentially, Transformers don’t process the words sequentially, but instead process each word independently in parallel by using attention.

The position of a word and the order of words in a sentence are important to understand the meaning of a text. To include this information, without having to process text sequentially, Transformers use positional encoding.

Understand positional encoding

Before Transformers, language models used word embeddings to encode text into vectors. In the Transformer architecture, positional encoding is used to encode text into vectors. Positional encoding is the sum of word embedding vectors and positional vectors. By doing so, the encoded text includes information about the meaning and position of a word in a sentence.

index

Statistical techniques were relatively good at Natural Language Processing (NLP) tasks like text classification. For tasks like translation, there was still much room for improvement.

A recent technique that has advanced the field of Natural Language Processing (NLP) for tasks like translation is deep learning.

When you want to translate text, you shouldn’t just translate each word to another language. You may remember translation services from years ago that translated sentences too literally, often resulting in interesting results. Instead, you want a language model to understand the meaning (or semantics) of a text, and use that information to create a grammatically correct sentence in the target language.

Understand word embeddings

One of the key concepts introduced by applying deep learning techniques to NLP is word embeddings. Word embeddings solved the problem of not being able to define the semantic relationship between words.

Before word embeddings, a prevailing challenge with NLP was to detect the semantic relationship between words. Word embeddings represent words in a vector space, so that the relationship between words can be easily described and calculated.

Word embeddings are created during self-supervised learning. During the training process, the model analyzes the cooccurrence patterns of words in sentences and learns to represent them as vectors. The vectors represent the words with coordinates in a multidimensional space. The distance between words can then be calculated by determining the distance between the relative vectors, describing the semantic relationship between words.

Imagine you train a model on a large corpus of text data. During the training process, the model finds that the words bike and car are often used in the same patterns of words. Next to finding bike and car in the same text, you can also find each of them to be used when describing similar things. For example, someone may drive a bike or a car, or buy a bike or a car at a shop.

The model learns that the two words are often found in similar contexts and therefore plots the word vectors for bike and car close to each other in the vector space.

Imagine we have a three-dimensional vector space where each dimension corresponds to a semantic feature. In this case, let’s say the dimensions represent factors like vehicle type, mode of transportation, and activity. We can then assign hypothetical vectors to the words based on their semantic relationships:

comptia training courses malaysia

Over the last decades, multiple developments in the field of natural language processing (NLP) have resulted in achieving large language models (LLMs).

To understand LLMs, let’s first explore the statistical techniques for NLP that over time have contributed to the current techniques.

The beginnings of natural language processing (NLP)

As NLP is focused on understanding and generating text, most first attempts at accomplishing NLP were based on using the rules and structure inherent to languages. Especially before machine learning techniques became prevalent, structural models and formal grammar were the primary methods employed.

These approaches relied on explicit programming of linguistic rules and grammatical patterns to process and generate text. Though these models could handle some specific language tasks reasonably well, they faced significant challenges when confronted with the vast complexity and variability of natural languages.

Instead of hard-coding rules, researchers in the 1990s began to utilize statistical and probabilistic models to learn patterns and representations directly from data.

exchange server certification training courses malaysia

Many online sites encourage users to share their views. People trust other people’s feedback about products, services, brands, and more. These comments are often frank, insightful, and seen to be free of marketing bias. But not all content is well intended.

Azure AI Foundry Content Safety is an AI service designed to provide a more comprehensive approach to content moderation. Foundry Content Safety helps organizations to prioritize work for human moderators in a growing number of situations:

Education

The number of learning platforms and online educational sites is growing rapidly, with more and more information being added all the time. Educators need to be sure that students aren’t being exposed to inappropriate content, or inputting harmful requests to LLMs. In addition, both educators and students want to know that the content they’re consuming is correct and close to the source material.

Social

Social media platforms are dynamic and fast moving, requiring real-time moderation. Moderation of user-generated content includes posts, comments, and images. Foundry Content Safety helps moderate content that is nuanced and multi-lingual to identify harmful material.

Brands

Brands are making more use of chat rooms and message forums to encourage loyal customers to share their views. However offensive material can damage a brand, and discourage customers from contributing. They want to be assured that inappropriate material can be quickly identified and removed. Brands are also adding generative AI services to help people to communicate with them, and therefore need to guard against bad actors attempting to exploit large language models (LLMs).

E-Commerce

User content is generated by reviewing products and discussing products with other people. This material is powerful marketing, but when inappropriate content is posted it damages consumer confidence. In addition, regulatory and compliance issues are increasingly important. Foundry Content Safety helps screen product listings for fake reviews and other unwanted content.

dynamics 365 training courses malaysia

Foundry Content Safety works with text and images, and AI-generated content.

Content Safety vision capabilities are powered by Microsoft’s Florence foundation model, which has been trained with billions of text-image pairs. Text analysis uses natural language processing techniques, giving a better understanding of nuance and context. Foundry Content Safety is multilingual and can detect harmful content in both short form and long form. It’s currently available in English, German, Spanish, French, Portuguese, Italian, and Chinese.

A severity level for each category is used to determine whether content should be blocked, sent to a moderator, or auto approved.

dynamics 365 supply chain training courses malaysia

Azure AI Foundry Content Safety is a set of advanced content moderating features that can be incorporated into your applications and services. Foundry Content Safety is available as a resource in the Azure portal.

Online content safeguarding is needed in a growing number of situations. Not only are we concerned with moderating content generated by people, but must also guard against the malicious use of AI.

Trusting user-generated content

Social interaction is increasingly a part of many digital spaces. Genuine user-generated content is seen as independent and trustworthy, and used alongside advertising and marketing. Different industries are encouraging their customers to connect with each other and their brand.

Harmful content has many negative effects. It damages trusted brands, discourages users from participating in online forums, and can have a devastating impact on individuals.

Foundry Content Safety is designed to be used in applications and services to protect against harmful user-generated and AI-generated content.

dynamics 365 sales training courses malaysia

In addition to tables, a relational database can contain other structures that help to optimize data organization, encapsulate programmatic actions, and improve the speed of access. In this unit, you learn about three of these structures in more detail: views, stored procedures, and indexes.

What is a view?

A view is a virtual table based on the results of a SELECT query. You can think of a view as a window on specified rows in one or more underlying tables. For example, you could create a view on the Order and Customer tables that retrieves order and customer data to provide a single object that makes it easy to determine delivery addresses for orders.

What is a stored procedure?

A stored procedure defines SQL statements that can be run on command. Stored procedures are used to encapsulate programmatic logic in a database for actions that applications need to perform when working with data.

You can define a stored procedure with parameters to create a flexible solution for common actions that might need to be applied to data based on a specific key or criteria. For example, the following stored procedure could be defined to change the name of a product based on the specified product ID.

dynamics 365 finance training courses malaysia

In a relational database, you model collections of entities from the real world as tables. An entity can be anything for which you want to record information; typically important objects and events. For example, in a retail system example, you might create tables for customers, products, orders, and line items within an order. A table contains rows, and each row represents a single instance of an entity. In the retail scenario, each row in the customer table contains the data for a single customer, each row in the product table defines a single product, each row in the order table represents an order made by a customer, and each row in the line item table represents a product that was included in an order.

Relational tables are a format for structured data, and each row in a table has the same columns; though in some cases, not all columns need to have a value – for example, a customer table might include a MiddleName column; which can be empty (or NULL) for rows that represent customers with no middle name or whose middle name is unknown.

Each column stores data of a specific datatype. For example, an Email column in a Customer table would likely be defined to store character-based (text) data (which might be fixed or variable in length), a Price column in a Product table might be defined to store decimal numeric data, while a Quantity column in an Order table might be constrained to integer numeric values; and an OrderDate column in the same Order table would be defined to store date/time values. The available datatypes that you can use when defining a table depend on the database system you are using; though there are standard datatypes defined by the American National Standards Institute (ANSI) that are supported by most database systems.

dynamics 365 field service training courses malaysia

You can use SQL statements such as SELECT, INSERT, UPDATE, DELETE, CREATE, and DROP to accomplish almost everything that you need to do with a database. Although these SQL statements are part of the SQL standard, many database management systems also have their own additional proprietary extensions to handle the specifics of that database management system. These extensions provide functionality not covered by the SQL standard, and include areas such as security management and programmability. For example, Microsoft SQL Server, and Azure database services that are based on the SQL Server database engine, use Transact-SQL. This implementation includes proprietary extensions for writing stored procedures and triggers (application code that can be stored in the database), and managing user accounts. PostgreSQL and MySQL also have their own versions of these features.

Some popular dialects of SQL include:

• Transact-SQL (T-SQL). This version of SQL is used by Microsoft SQL Server and Azure SQL services.
• pgSQL. This is the dialect, with extensions implemented in PostgreSQL.
• PL/SQL. This is the dialect used by Oracle. PL/SQL stands for Procedural Language/SQL.

Users who plan to work specifically with a single database system should learn the intricacies of their preferred SQL dialect and platform.

dynamics 365 customer service training courses malaysia