Coding with Qwen 2.5: An OverviewPicture by Editor (Kanwal Mehreen) | Canva

 

Giant language fashions (LLMs) have been utilized in many purposes, and plenty of firms are racing to make use of one of the best fashions out there for his or her aggressive benefits. This, in flip, makes many open-source initiatives develop one of the best mannequin in comparison with the others.

One of many fashions to look at for is the Qwen household mannequin, developed by the Alibaba crew. They intention to give attention to constructing open supply generalist fashions, together with giant language and multimodal fashions, that would compete even towards closed supply fashions.

With the discharge of their newest mannequin, Qwen2.5, the crew has proven that their mannequin has considerably improved in lots of areas. With a lot potential, we are going to check out the mannequin ourselves.

This text will discover Qwen2.5 and learn how to use it in your work.

 

Qwen2.5 Mannequin Household

 
As talked about, Qwen2.5 is the most recent collection of Qwen fashions from the Alibaba group. Seven totally different parameters can be found, together with 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B, in addition to base and instruct variants. Furthermore, the mannequin can assist longer contexts of as much as 128k tokens and generate as much as 8k tokens with assist for 29 totally different languages.

The discharge can be accompanied by the specialised coding and arithmetic fashions Qwen2.5-Coder and Qwen2.5-Math, respectively. Each are helpful fashions for particular use circumstances and can be found instantly through HuggingFace.

Let’s check out the mannequin ourselves. First, we have to set up the library used for this code instance.

pip set up transformers optimum auto-gptq qwen-vl-utils flash-attn --no-build-isolation

 

We are going to use the PyTorch frameworks, so choose the one that’s most acceptable to your setting.

Let’s attempt utilizing the pipeline transformers for textual content era. On this instance, we are going to use the Quantized 7B Instruction variant.

from transformers import pipeline
messages = [
    {"role": "user", "content": "Who are you in one sentence?"}
]
pipe = pipeline("text-generation", mannequin="Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4", system=0)
outcome = pipe(
    messages,
    max_length=100,        
    temperature=0.7,      
    top_p=0.9,            
    repetition_penalty=1.2  
)

print(outcome)

 

Output:

'I'm Qwen, an AI assistant created by Alibaba Cloud designed to assist customers generate varied sorts of textual content content material.'

 

We will see from the outcome above that the mannequin can comply with the immediate directions effectively. Let’s check out with a immediate that pushes for extra creativity. We are going to change the parameters a bit of bit to lower the token measurement and enhance the temperature.

messages = [
  {"role": "user", "content": "Write a short story about a robot who learns to paint landscapes."}
]
outcome = pipe(
    messages,
    max_length=200,        
    temperature=0.8,      
    top_p=0.9,            
    repetition_penalty=1.2  
)

print(outcome)

 

Output:

'Within the bustling metropolis of Neo-Tokyo, there lived an unassuming robotic named Pixel. Not like his fellow robots within the manufacturing unit the place he was born, Pixel had at all times been fascinated with artwork and nature. His mechanical arms had been usually seen shifting rhythmically as if portray invisible strokes on air'

 

As we will see from the output above, the mannequin can creatively create a easy story. If we set the max_length to a better quantity, then the story will develop into for much longer.

Lastly, we will check with totally different roles, such because the system and person, to raised element the outcome.

messages = [
  {"role": "system", "content": "You are a helpful and concise assistant who uses simple language."},
  {"role": "user", "content": "Explain how photosynthesis works."}
]

 

Output:

Certain! Photosynthesis is the method by which crops, algae, and a few micro organism make their very own meals utilizing daylight. Right here’s a simplified breakdown of the way it works:nn1. **Gentle Absorption**: Vegetation have inexperienced pigments referred to as chlorophyll in constructions referred to as chloroplasts inside their cells. Chlorophyll captures gentle vitality from the solar'

 

The result’s adequate, particularly with the system position, after we can change the fashion as we wish.

Subsequent, we are going to check out varied fashions in Qwen-2.5. To make it simpler, we are going to create a operate to name the mannequin.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

def qwen_model_testing(model_name, immediate):


  mannequin = AutoModelForCausalLM.from_pretrained(
      model_name,
      torch_dtype="auto",
      device_map="auto"
  )
  tokenizer = AutoTokenizer.from_pretrained(model_name)


  messages = [
      {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
      {"role": "user", "content": prompt}
  ]

  textual content = tokenizer.apply_chat_template(
      messages,
      tokenize=False,
      add_generation_prompt=True
  )
  model_inputs = tokenizer([text], return_tensors="pt").to(mannequin.system)


  generated_ids = mannequin.generate(
      **model_inputs,
      max_new_tokens=512
  )
  generated_ids = [
      output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
  ]

  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
  print(response)

 

First, we are going to use the smaller instruction parameter variants to see how it’s performing. We may also attempt zero-shot prompting to instruct the mannequin as a sentiment analyzer mannequin.

model_name = "Qwen/Qwen2.5-Coder-1.5B-Instruct-GPTQ-Int4"
immediate = """Classify the next sentence as both "Constructive" or "Unfavorable" in sentiment:

  "General, I would not suggest this product to anybody."
  Output:"""

qwen_model_testing(model_name, immediate)

 

Output:

 

Then, we are going to check out the Coder variant, which is the mannequin that was skilled particularly for code era, code refinement, and code evaluate. On this instance, we are going to attempt to generate Python whereas asking them to clarify the ideas.

model_name = "Qwen/Qwen2.5-Coder-7B-Instruct-GPTQ-Int4"
immediate = "Clarify the time complexity of the binary search algorithm and supply a brief Python code snippet that implements it."

qwen_model_testing(model_name, immediate)

 

Output:

The time complexity of the binary search algorithm is O(log n), the place n is the variety of parts within the sorted array on which the search is carried out. It is because binary search works by repeatedly dividing the search interval in half. If the worth of the search secret's lower than the merchandise in the course of the interval, slim the interval to the decrease half. In any other case, slim it to the higher half. Repeatedly verify till the worth is discovered or the interval is empty.

Right here is a straightforward Python code snippet implementing binary search:

```python
def binary_search(arr, goal):
    low = 0
    excessive = len(arr) - 1
    
    whereas low  goal:
            excessive = mid - 1
        else:
            low = mid + 1
            
    return None

# Instance utilization:
arr = [1, 3, 5, 7, 9]
goal = 5
outcome = binary_search(arr, goal)
if outcome just isn't None:
    print(f"Ingredient discovered at index {outcome}")
else:
    print("Ingredient not discovered")
```

On this code, `binary_search` takes an array `arr` and a `goal` worth as inputs. It initializes two pointers, `low` and `excessive`, to signify the present search interval. The loop continues till `low` exceeds `excessive`. In every iteration, it calculates the center level `mid`, checks if the center component is the goal, and adjusts the search interval accordingly. If the goal just isn't discovered, it returns `None`

 

The result’s fairly good, with the reason ample with the instruction given.

Lastly, we are going to attempt the Math variant, which is the specifically skilled Qwen mannequin for the mathematical reasoning course of.

model_name = "Qwen/Qwen2.5-Math-7B-Instruct"
immediate = "Discover the following quantity on this sequence and clarify your reasoning: 2, 9, 30, 93, ..."

qwen_model_testing(model_name, immediate)

 

Output:

To seek out the following quantity within the sequence 2, 9, 30, 93, ..., we have to determine the sample within the sequence. Let's study the variations between consecutive phrases:

- The distinction between the second time period (9) and the primary time period (2) is (9 - 2 = 7).
- The distinction between the third time period (30) and the second time period (9) is (30 - 9 = 21).
- The distinction between the fourth time period (93) and the third time period (30) is (93 - 30 = 63).

Now, let us take a look at the sequence of those variations: 7, 21, 63. We discover that every time period on this new sequence is thrice the earlier time period:

- (21 = 7 occasions 3)
- (63 = 21 occasions 3)

Following this sample, the following time period within the sequence of variations needs to be (63 occasions 3 = 189).

To seek out the following time period within the authentic sequence, we add this distinction to the final time period of the unique sequence:

[93 + 189 = 282]

Due to this fact, the following quantity within the sequence is (boxed{282}).

 

The mannequin is ready to purpose effectively with the given mathematical sequential downside, which exhibits that it has the potential to assist varied issues sooner or later.

You possibly can try the Qwen Documentation for additional studying.

 

Conclusion

 
Qwen 2.5 is the latest addition from the Qwen household by the Alibaba group. Seven totally different parameters can be found with base and instruction variants. There are additionally coder and math-specialized fashions that had been launched for the Qwen 2-5 household.

On this article, we use instance code to display learn how to use the Qwen 2.5 mannequin and its variants.

I hope this has helped!
 
 

Cornellius Yudha Wijaya is a knowledge science assistant supervisor and knowledge author. Whereas working full-time at Allianz Indonesia, he likes to share Python and knowledge ideas through social media and writing media. Cornellius writes on a wide range of AI and machine studying subjects.

Leave a Reply

Your email address will not be published. Required fields are marked *