About The Role
We are seeking a Data Scientist specializing in natural language processing (NLP) and prompt engineering to help build an AI-powered service focused on intelligent text processing and recommendation features. You'll be responsible for developing NLP models and optimizing prompt interactions with large language models (LLMs) to enhance user experiences and drive insights across our platforms.
Key Responsibilities
- NLP Model Development:
- Develop and fine-tune NLP models to extract, normalize, and structure information from unstructured text sources such as resumes, job descriptions, and user-generated content.
- Implement text classification, entity recognition, and other NLP downstream tasks to improve the relevance and quality of recommendations.
- Prompt Engineering for LLMs:
- Design and optimize prompts to enhance LLM performance, improving context-aware responses and information retrieval.
- Experiment with various prompt structures and techniques to optimize for specific tasks, such as question answering, content generation, and conversational AI.
- Data and Model Pipelines:
- Work closely with data engineering teams to establish data pipelines that support efficient training and deployment of AI Models.
- Build and maintain model deployment pipelines, ensuring models are tested, validated, and continually improved.
- Performance Monitoring and Optimization:
- Track and analyze model performance, conducting A/B testing and using evaluation metrics to optimize results.
- Implement feedback loops to continuously improve model accuracy and prompt efficacy.
- Documentation and Collaboration:
- Document model specifications, prompt experiments, and workflows to maintain clarity and facilitate collaboration.
- Collaborate with product managers and engineering teams to ensure the AI service aligns with product needs and user expectations.
Requirements
- Education: Bachelor's degree in Computer Science, Information Technology or a related field.
- Experience: 2+ years of experience in NLP and AI
- Technical Skills:
- Strong proficiency in Python and NLP libraries (e.g., SpaCy, NLTK, Hugging Face Transformers).
- Experience in prompt engineering to optimize language model responses.
- Hands-on experience with ML frameworks such as TensorFlow or PyTorch for NLP tasks.
- Proficiency in RESTful API development and deployment on cloud platforms (e.g., AWS, Google Cloud).
- Analytical Skills: Skilled in evaluating and improving model and prompt performance, with a strong understanding of NLP evaluation metrics.
- Communication: Ability to document and explain technical details clearly to collaborate effectively with cross-functional teams.
Preferred Qualifications
- Experience with vector databases (e.g., Pinecone, Elasticsearch) for semantic search.
