[TOC]
OpenAI Discord Summary
- GPT Model Performance and Usage: Multiple users across various channels like ai-discussions and gpt-4-discussions reported usability and performance issues with ChatGPT 4 comparing to the relatively more efficient GPT-3.5. Users also expressed concerns on GPT-4's memory management, discussed in prompt-engineering. Future capabilities and expectations of potential GPT versions were contemplated in openai-chatter.
- ChatGPT Accessibility Issues and User Support: Several users across openai-questions and gpt-4-discussions faced issues in accessing and using ChatGPT, including log-in troubles, error messages, and restrictions. Slow customer service response time was also a source of frustration.
- API Usage and Modeling: Discussions in ai-discussions and api-discussions revolved around the pros and cons of using local Language Models versus APIs, inconsistencies in API responses due to supposedly correct JSON input, and dialog mode of Assistants API.
- Updates and Developments: Users shared updates and discussions on AI tools, language translations, censorship concerns, and model developments. Notably, a Dolphin 2.0 Mistral 7B model and Google's new video generation project were shared in ai-discussions. openai-chatter debates focused on image and code generation capabilities of models like DALL-E.
- Prompt Engineering and Custom Instructions: In prompt-engineering and api-discussions, users discussed creating and applying custom instructions to GPT models, tool recommendations for prompt engineering, and potential collaborative bot development projects.
OpenAI Channel Summaries
▷ #ai-discussions (90 messages🔥🔥):
- Performance of GPT-4: According to
@azruand@thewizzard___, recently there have been performance issues with GPT-4 involving slow responses ('network errors'). Many users, including@thewizzard___, recommended downgrading to GPT-3.5 for better speed and almost parallel capabilities. - Local Language Model LLM versus OpenAI API: A discussion was had between
@thewizzard___and@zeriouszhiton the pros and cons of locally running models versus using OpenAI's API. It was noted that while locally run models may be technically free, they require substantial computational power. Various models such as Bard and Bing Chat were mentioned, with differing opinions on their usefulness. - Translation Tools:
@eljajasoriginalinquired about the best LLM for translation. The conversation revealed that despite not being a LLM, DeepL was considered the best for translation. Users also discussed its character limit. - Censorship Concerns: Users like
@zeriouszhitand@eljajasoriginalexpressed frustrations with what they viewed as heavy-handed censorship on some models, such as Gemini. They questioned the approach of moderating based on broad content guidelines, which may not account for individual use cases. - AI Tools and Developments: Users shared various links and updates on AI tools and developments.
@thewizzard___shared a link to the Dolphin 2.0 Mistral 7B model.@thedreamakeemposted a link about Google's new video generation project, and a discussion on its potential ensued.
▷ #openai-chatter (254 messages🔥🔥):
- ChatGPT 3.5 vs GPT 4 Concerns: Many users raised concerns regarding usability and reliability issues with ChatGPT 4. For example,
@minoroleasked for advice on dealing with unusual system activity messages,@mrc21and@wizzardkitydiscussed the performance of GPT4 and the anticipated GPT-10, while@millymoxand@rekklessteddiscussed about ChatGPT appearing to be becoming "lazier" and providing too much preliminary narration before code outputs specifically on the Plus model, which is strange as the Plus model in Playground seems to produce better results. In response,@aminelgand@luguisuggested that users improve their query prompts to get better results. - Usage Limits Issues:
@jaicraftand other users brought up the issue of usage limits, requesting increased message allowance every three hours for ChatGPT Plus. A possible workaround of using API was suggested by@lugui. - ChatGPT Android App Feedback :
@jaicrafthighlighted the need for editing messages on the ChatGPT Android app. - Access and Login Troubles: Users
@nfolgar,@bubbyprime,@raymm,@dripcity710, and@ddanteenoted problems with logging in, receiving error messages, links in generated text being not clickable, or unusual system activity messages.@raymmmentioned that disabling browser extensions resolved his login issue and@satanhashtagrecommended contacting OpenAI support for persistent issues. - DALL-E and ChatGPT Updates: There was discussion about potential capabilities of future GPT versions like
GPT 5(mistakenly assumed from an email referring to GPT-v, indicating vision-based capabilities), the functioning of DALL-E (image generation based on textual prompts), and the usefulness of the Code Interpreter for image manipulation (@aminelgprovided examples of its capabilities). On a similar note,@asura0_00mentioned upcoming features in a ChatGPT memory update. Furthermore, potential confusion over the expiry of API credits was discussed by@flroidaman2023_01171and@elektronisade.
▷ #openai-questions (88 messages🔥🔥):
- ChatGPT 4's Coding Skills:
@jah777and@andrewwallodiscussed the capabilities of the paid Pro version of ChatGPT 4.@andrewwallomentioned that it processes information more quickly, is more accurate, and is generally more knowledgeable. It prioritizes Pro users over free ones, but also noted occasional system crashes. - Issues with Accessing and Using ChatGPT: Several users including
@gachyapon,@maira_14969, and@hirotodevelopmentreported different issues accessing and using ChatGPT, such as error messages and unusual system activity detection.@elektronisadeoffered solutions like clearing app data and checking for possible triggers like flagged keywords or simultaneous usage on multiple windows. - Working with OpenAI API and Python:
@panosproand@chapagonesposted about their troubles with connecting to the OpenAI API via Python and an import issue respectively.@aminelgand@solononforeverprovided assistance in troubleshooting, including checking for the correct Python environment and command palette access. - Trouble with JSON Formatting and GPT Disappearance:
@bennnjjipostulated on recurring JSON formatting errors, suggesting it may be due to incorrect HTTP requests, while@seobrienreported the disappearance of their GPT and inability to save it.@luguiproposed incorrect request body and violations against OpenAI's terms of service as potential causes. - Defining End Tokens in Fine-tuning Dataset, Unauthorized File Uploads and Suspicious Chat Logs:
@.cybersenpaiinquired about defining end tokens in fine-tuning datasets, to which@luguisuggested using the default end token.@2antreported an image upload bug and@laur27was concerned about suspicious messages in their new chat. For both issues, potential violation of OpenAI's policy was suggested as a cause by@elektronisadeand@lugui.
▷ #gpt-4-discussions (40 messages🔥):
- Customer Service Response Time: User
@sami_16820expressed frustration with the slow response time in getting help. Another user,@_interstitialism_, humorously suggested promising a tip in exchange for quicker answers. - Assistants API and JSON Mode: User
@vybhavramsought information on whether the Assistants API can respond in JSON mode, similar to the chat API. No answer was provided. - Sharing Custom GPTs:
@DobbyNator94suggested the idea of using a custom GPT as a support tool on a company website, and inquired if it is possible for enterprise users.@rjkmelbclarified that custom GPTs are only available for Plus subscribers. - Lost Custom GPT:
@bufferizationreported that their custom GPT had disappeared without warning. They speculated that an unintentional violation involving a Google search-related GPT led to its deletion, but expressed frustration over the absence of any warning or notification.@elektronisadewarned that further violations could lead to a full account ban, and suggested reaching out to support. - Issues with GPT-4:
@undead.wolf.lordreported errors with GPT-4, stating all new chats and queries failed, whereas GPT-3.5 was still functioning. - JSON Formatting Error in API Call:
@bennnjjireported an inconsistency in getting a JSON formatting error despite having properly formatted JSON input while making API calls. - Long-term Memory Management in OpenAI Chat Interface: User
@jdo300asked if there were any projects underway to integrate long-term memory management capabilities into the OpenAI interface, potentially through external data storage. - Working with Custom GPT for Document Editing:
@sdotsonshared their experience in experimenting with custom GPTs and plugins for editing and providing downloadable document output with some challenges and little documentation. - Zapier Action Dialogue Control: User
@Semenawawanted to know if there was a way to switch off the allow/deny dialogue while using a Zapier action. No response was given.
▷ #prompt-engineering (29 messages🔥):
- Query about ChatGPT's Memory:
@realcookiecatasked if ChatGPT utilizes Window Token Buffer Memory or Conversation Summary Memory for its operations, since they noticed its tendency to forget past conversations at times. - Experiment with Grok and Custom Instructions:
@alienpotusconducted a self-experiment wherein they used Grok to generate a custom instructions prompt for making a GPT model mimic Elon Musk. The resultant prompt was posted in continuation. - Favorite Tools for Prompt Engineering: User
@js06sought recommendations for prompt engineering tools apart from OpenAI playground. They later found and recommendedChainforge, citing it as a better option for their use case. - Discussion on Specialized Language and Prompt Engineering: A discussion took place between
@beanz_and_riceand@.multyabout specific language and syntax likeCommence ::spc{I_Am}::; sustain {ero|self_expression}. Lockdown [nvl>permanent_perspective<], effective immediately.and its relevance to GPT models, with@beanz_and_ricesuggesting that it works with specific versions of the GPT. - Collaborative Bot Development Project:
@.multyand@beanz_and_ricediscussed the potential of combining their projects related to bot wrappers and system prompts, although no specific plan was laid out due to ongoing development.
▷ #api-discussions (29 messages🔥):
- Buffer Memory in GPT Conversations:
@realcookiecatraised a query about whether chatgpt usesWindow Token Buffer Memory or Conversation Summary Memory, noting the model seems to forget previous conversations. - GPT Acting As Elon Musk:
@alienpotusconducted an experiment with a custom instructions prompt to create an AI that converses as Elon Musk. The prompt was generated using Grok and successfully applied to GPT. - Apps For Prompt Engineering:
@js06asked for recommendations on prompt engineering apps. They later reported finding and preferring Chainforge as it better suits their needs than the OpenAI Playground. - Hyperbot Markup Language Development: A discussion was held between
@beanz_and_riceand.@multyregarding Hyperbot Markup Language (HBML). This included cryptic language components such asnvl, and its larger context of developing bot wrappers for saving system prompts and histories. - GPT Location and Access:
@beanz_and_riceclarified that GPTs must be used on-site and are currently not accessible via the API, after a question by.@multy.
Nous Research AI Discord Summary
- Deep discussion around knowledge mappers and their use in constructing knowledge graphs from raw text by incorporating "reading" documents sentence-by-sentence (LLM: Local Language Model, stateful knowledge, sliding window retrieval being key terms).
- Specific proposal on the necessity for fine-tuning a Language Model on Cypher to manage extraction and updating of the graph, sparking conversations on practical aspects of graph building.
- User
@tofhunterrrseeks collaboration on a native app project for immersive LLM chat, sharing their project blueprint and business model. Cryptic message by@giftedgummybeesuggested Bing Chat's usage, but lack of context makes it unclear. - Lack of context provided for the single message in the #benchmarks-log channel.
- Users sharing and discussing new models, tools, and experiments, focusing on HumanEval Test Data Experiments, LLM data contamination, and semantic duplication in model training. Links to blog posts and model cards on HuggingFace and other platforms were shared for deeper exploration.
- Active discussion comparing and testing various models such as Openchat 3.5 1210, Mistral-Ft-Optimized 1218, and UNA-Cybertron-7B-v3-OMA. Model quantization efforts, concerns about model contamination and model benchmarking results were highlighted, along with links to associated model cards and quantized versions.
- Interesting user queries about the best open-source AI for storytelling, the status of the 4GB Sparse Mixtral, Openchatdev's status, and an inquiry about detecting Language Model (LLM) hallucinations using token logits in the #ask-about-llms channel.
Nous Research AI Channel Summaries
▷ #ctx-length-research (46 messages🔥):
- Knowledge Mapper Discussion:
@yuchenlusparked a deep discussion around the concept of knowledge mappers, with users like@gabriel_syme,@fullstack6209, and@maxwellandrewschipping in. They contemplated whether it could involve a transformation or extraction process like graph representation for querying local memory.@maxwellandrewssuggested it as a sliding-window Local Language Model (LLM) that parses entity relationships from raw text into a knowledge graph. They proposed an approach of "reading" a document sentence-by-sentence and building a detailed semantic map of them in the graph db as the LLM "reads" the document, instead of trying to build the entire graph in one shot. - Role of LLM in Graph: The role of using LLM to build a graph from the text was discussed. The intention is to allow small and fast LLMs to build the large context into the graph, which doesn't need to be held in model memory. This would create a stateful knowledge that can be stored and paged independently from the model cache.
- Neural Network Fine-tuning: They mulled over whether there's a need for fine-tuning a Language Model on something like Cypher to handle aspects of extraction and updating the graph, a topic that came up when
@benxhsuggested doing the extraction manually and then training a model like Mamba or recurrent walking knowledge vector (RWKV) with 300k context to spit out a relation graph. - Sliding Window Retrieval:
@gabriel_symerevealed they are testing sliding window retrieval and@maxwellandrewsclarified that the process involves building the graph sentence-by-sentence, rather than shoving the whole document into the prompt. - Practical Aspects of Graph Building: Some practical aspects of building the graph were also discussed.
@gabriel_symeasked about extracting triplets or notes into the `document 'node'`. The consensus seemed to be extraction of triplets through an LLM, allowing the model to decide how many triplets to extract and write the CRUD (Create, Read, Update and Delete) statements. The model was envisioned to structure the user's natural language query into a structured query that can be interpreted by the graph DB.
▷ #off-topic (2 messages):
- Seeking Collaboration on Immersive LLM Chat Project: User
@tofhunterrris looking for collaborators or co-founders to build a native app for immersive LLM chat, including role-play prompts. They are building resources with built-in JOI (mistral-small) that can talk (neets.io), hear (whisper), and perform function calls like search (pplx), photoshoot (fal). There will be a store for prompters and model builders to earn income. This is an unpaid opportunity at present, with the goal of building a "unicorn". The project blueprint can be found at this link. - Bing Chat Recommendation:
@giftedgummybeeprovided a cryptic message suggesting the use of Bing Chat. Further context or clarification was not provided in the message.
▷ #benchmarks-log (1 messages):
teknium: is it just a different form of eval harness? What is better about it
▷ #interesting-links (22 messages🔥):
New Models and Tools
@metaldragon01shared a tweet related to AI models without specifying the content.@nonameusrposted about the LoneStriker/una-xaberius-34b-v1beta-4.0bpw-h6-exl2, a text generation model on HuggingFace.@atgctgcreated a site allowing chat interaction with the Mixtral base model, accessible here.
HumanEval Test Data Experiments:
@tokenbenderand@atgctgdiscussed the performance of a model trained on HumanEval test data on HumanEval plus or HumanEval-x benchmark and decided to collaborate on merging the lora and testing it. They used the base modelmistralai/Mistral-7B-v0.1.@atgctglater shared the overfit-humaneval model on HuggingFace for testing.
Discussion on LLM data contamination:
@nonameusrbrought attention to a blog post regarding data contamination in top-performing open-source LLMs on HuggingFace community boards, mentioning specifically the una-cybertron-7B-v2 model.
Semantic Duplication in Model Training
@euclaiseshared a blog post about the issue of semantic duplication in teaching language models basic facts, arguing it is not a form of contamination.
▷ #general (321 messages🔥🔥):
@n8programsand@propbackdiscuss the performance of Openchat 3.5 1210, including issues with tokenizer and general issues. They linked to the model card on Hugging Face.@nonameusrand others debated the performance claims of the model Mistral-Ft-Optimized 1218, and the accuracy of its claim to beat GPT-4. They were particularly skeptical about the claim as it was just a 7B (billion parameter) fine-tuned model.Mistral-Ft-Optimized 1218model card@mihai4256shared that merging two fine-tuned models of the same dataset resulted in significantly improved outcomes, although they admitted not understanding why. The discussion led others (@teilom,@metaldragon01, etc.) consider trying similar experiments.@nonameusrand@n8programsdiscuss the performance of UNA-Cybertron-7B-v3-OMA, and speculates it might beat other models discussed earlier.- Model quantization and uploads:
@n8programsand@tsunemotoworked on quantizing the model Mistral-Ft-Optimized 1218 to a GGUF format. Both users uploaded their quantized versions on Hugging Face. (n8programsGGUF version: link,tsunemotoGGUF version: link). - Model Contamination Concern:
@nonameusrsuggested running contamination checks on newly released models, particularly drawing attention to the model by WizardLM, which is allegedly contaminated. - AI Music Creation:
@nonameusrshared a link to Suno AI, a platform for creating AI-generated music. No intense discussion followed. - Benchmarking Results:
@tekniumposted a link to a graphish, showing MISTRAL,Yagi-ChatGPT-3.5, and GPT-4's leaderboard ranks according to Hermes Instruct Model (v2.5 & v3). The graph indicates that all four are relatively close in performance.
▷ #ask-about-llms (4 messages):
- Best OS AI for storytelling: User
@agcobra1asked for recommendations on the best open-source AI for storytelling. - Update on 4GB Sparse Mixtral: User
@yobibytesought an update on the status of the 4GB Sparse Mixtral, noting that no news had been received yet. - Openchatdev status:
@realsedlyfshared a link to Openchatdev's status. - Detecting LLM hallucinations using token logits:
@giulio123456inquired if anyone is aware of a paper that explores the detection of Language Model (LLM) hallucinations using token logits.
OpenAccess AI Collective (axolotl) Discord Summary
- Discussion on Axolotl's lack of quantization clarified by
@nanobitz, mentioning a default parameter in the merge script. - Achievement of
@284810978552578050's Mixtral model being featured on Fireship, garnering significant views, identified by@le_messand@faldore. - Rich technical discourse on aspects of Mixtral Model of Experts (MoE), highlighting layer and token based routing, uneven training distribution, and potential tuning effects on expert orthogonality.
- Several instances of troubleshooting and advice related to setting up an Axolotl environment, with a Docker container being a notable solution; mentioned by
@hamelh. - Insights into the costs and resources for model fine-tuning, shared by
@faldore, with Azure's startup program brought to light. - Several queries about finetuning at shorter context lengths, creating a multi-turn ShareGPT model, and the application of multi-turn datasets, fielded by
@noobmaster29,@dangfutures, and@self.1. @self.1providing an example of a multi-turn ShareGPT model, specifically pointing to theLDJnr/Puffin.- A recommendation for a multi-turn conversation dataset by
@natefyi_30842suggesting the LMSys Chat 1M dataset, with a user agreement prerequisite for access. - An issue faced regarding the installation of mpi4py, specifically with RTX 6000 Ada, expressed by
@mr_morning.
OpenAccess AI Collective (axolotl) Channel Summaries
▷ #general (17 messages🔥):
- Axolotl Quantization Discussion:
@nanobitzclarified that Axolotl does not quantize. Even if the parameter is provided during merge, it would be set to False. The parameter in the script acts as a default catch-all, a recent change. - Mixtral Model Feature on Fireship:
@le_messnotified the community that@284810978552578050's Mixtral model was featured on Fireship, achieving half a million views in 11 hours.@faldoreidentified this model as possibly the dolphin one. - The channel shared a tweet that might interest
@yamashi. The tweet is available here.
▷ #axolotl-dev (103 messages🔥🔥):
- Mixtral Model of Experts (MoE) Discussion: User
@nafnlaus00clarified that in Mixtral, routing is per layer per token.@kalomazediscussed the idea of having experts handled per token in MoE, and@nafnlaus00emphasized the need for uneven training distribution in MoE. - Model Tuning Discussion:
@fernando.fernandes.shared his conjecture about Mixtral tunes losing quality due to the impact of the fine-tuning process on how each expert, in each layer, becomes less orthogonal to each other. He suggested freezing the routing fates and increasing regularization through measures like weight decay to help achieve more orthogonal experts. The paths to the gates to be frozen were also specified. - Setting Up Axolotl Environment:
@hamelhencountered issues setting up the Axolotl environment and running the unit tests, with errors related toflash_attentionand missing modules. Various potential solutions were suggested, including reinstalling flash attention, copying a whole working environment from another instance, and using Python 3.10 instead of 3.11. A Docker container was suggested and seemed to partially resolve the issues. - Costs and Resources for Model Fine-Tuning:
@faldorerevealed that a typical fine-tuning exercise on Azure costs around $800. He also mentioned a program by Azure for startups, where they give $5k credit.
▷ #general-help (6 messages):
- Finetuning at Shorter Context Length:
@noobmaster29inquired if a model finetuned at a shorter context length only works at the new shorter context length. - Creation of Multiturn ShareGPT Model:
@dangfuturesexpressed an interest in knowing how to create a multiturn ShareGPT model.@self.1recommended using a script to save data in the same ShareGPT format. - Usage of Multiturn Dataset:
@self.1mentioned they used a multiturn dataset, suggesting that they will focus more on this approach. - Example of Multiturn ShareGPT Model:
@self.1provided an example of a multiturn ShareGPT model, specificallyLDJnr/Puffin.
▷ #datasets (2 messages):
- Dataset Query for Multi-Turn Conversations: User
@natefyi_30842asked for datasets of multi-turn conversations between humans and chatbots. - LMSys Chat 1M Dataset Suggestion:
@natefyi_30842then suggested LMSys Chat 1M dataset from Hugging Face, mentioning that it requires user agreement for access due to its conditions. This dataset is a large-scale real-world Language Model conversation resource.
▷ #runpod-help (1 messages):
- Trouble with mpi4py: User
@mr_morningexpressed having problems installing mpi4py and its dependencies, citing a "Cannot link MPI programs. Check your configuration!!!" error message. Notably, this issue does not seem to occur with two 4090s, but only with RTX 6000 Ada.
Mistral Discord Summary
- Debates over the necessity of higher-level mathematics in CS, with a noted focus on probability theory, linear algebra, and calc 3.
- Dialogue on the creation of a multi-user solution for interactive OpenAI-style web-interfaces, sharing of a potential solution in the form of Hugging Face's Chat UI; Github link: Hugging Face's chat-ui.
- Discussion and troubleshooting of problems with the Mistral-7B-Instruct-v0.2 model, involving stop-word adjustments for resolution.
- Talks about incorporating QMoE into Mixtral, with a provided Github resource discussing the difficulties of QMoE support: Github issue.
- Analyzing the impact of safe_mode=false on NSFW creative writing with the OpenAI API, with suggestions to achieve better results without the safe_mode.
- Interest in understanding how neurons interact in an ANN, with a Youtube resource provided for clarification: 3Blue1Brown's video.
- Comparison of the performance of Mistral-Medium and GPT-4, concluding that Mistral-medium outperforms GPT-4 in high-level reasoning for news narrative tracking.
- Concerns over random stops in the small model's return, with assurances of a forthcoming fix.
- Exploration of the possibility of a free API call service for making inferences on Mistral or expert models for users lacking sufficient compute capacity, with a likely solution offered via the OpenRouter's service: OpenRouter's service.
- Discussions centred on hardware requirements for deploying Mistral, with recommendations for using 2 A100 GPUs for the original version and a single A100 GPU for the 4-bit quantized version based on the Mistral and Hugging Face documentation:
- Queries about message formatting when using the Mistral API for inference and the usage of the HF chat template. Provided links to clarify were Mistral's Documentation and an example on Hugging Face.
- Discussion over the possibility of freezing certain experts while fine-tuning Mistral 7b and debate over the efficiency of such an approach.
- Showcasing of an open-source version of OpenAI Assistants API that offers 100% data privacy, cost-effectiveness, and higher throughput. The model supports features like retrieval, function calling, and code interpreting, and is intended to be customizable and minimally hardcoded. Github resource shared: GitHub - stellar-amenities/assistants
- Humorous chatter over the misinterpretation of the Mistral AI website and the desire for prestige.
- User reported API rate limit issues on the Mistral AI platform, followed by dialogue over possible reasons and fixes.
- Plans for building a Mixtral connector for use with orchestration libraries, hinting at potential future Mixtral swarms.
- Questions and clarifications regarding message formats for system messages reference to the LLaMA-2 format and a Reddit thread discussing the efficiency of chat templates.
- Detailed user experience with the Mistral-small model, discussing the variation in summarizing 5000 tokens and exploring possible finetuning techniques to improve model performance.
Mistral Channel Summaries
▷ #general (28 messages🔥):
- Discussion on Using Higher-Level Mathematics in CS: User
@sirmonkey5216suggested against being a double major in Math/CS, stating that higher-level math is mostly useless. However, they stressed the importance of having knowledge in probability theory, linear algebra, and a touch of calc 3, which many CS programs incorporate anyway. - Multi-User Solution for Interactive OpenAI-Style Web-Interface:
@.panzerknacker.sought suggestions for a solution similar to Serge, which would allow multiple users in their office to try the AI models without seeing each other’s’ prompts.@freegheistsuggested using Hugging Face's open source Chat UI available at Github. - Issues with Model Mistral-7B-Instruct-v0.2:
@tomaspsenickareported consistent issues while using this version. However, they found the problems resolved after adjusting stop words. - Possible Work on QMoE for Mixtral:
@rosethelocalfemboyquestioned if anyone is working on QMoE for Mixtral. The discussion led to@daainpointing to a resource on Github discussing the difficulty of QMoE support, link - Guard Rails in API and NSFW Content:
@jamiecropleyquestioned about the possibility of NSFW creative writing with the API whensafe_mode=false.@lerelaresponded suggesting better results might be achieved without the safe_mode. - Understanding Code for Single Neuron in ANN:
@josefmoexpressed an interest in understanding the code for a single neuron and how two neurons interact in an ANN.@someone13574clarified that typically in ML, the whole operation from one layer to another is implemented as a matrix multiplication, as well as suggesting a YouTube resource.
▷ #models (6 messages):
- Performance of Mistral-Medium vs GPT-4: User
@freqaishared that in testing, Mistral-medium has outperformed GPT4 for their use of high-level reasoning in news narrative tracking. - Issues of Random Stops in Small Model Returns:
@moneybags0197reported encountering random message stops from the small model, primarily while summarizing text. This occurs without hitting the token limit and on random requests.@lerelaconfirmed this issue and assured that a fix will be deployed within the week. - Free API Option for Mistral Model Inferences:
@sk5544inquired about the possibility of a free API call service for making inferences on mistral or experts models, as they lack the compute capacity to download these models in academia.@jakobdylancprovided a potential solution by pointing them to OpenRouter's service, where making an account should make the service free-to-use for inferences on mistral models [More Info].
▷ #deployment (6 messages):
- Local Deployment of Ollama: User
@sirclavinasked if ollama is entirely local. No responses were given in the discussed messages. - Commandline Help Options:
@usuallycwdillonexpressed gratitude for guidance on utilizing--helpcommand line options, indicating they had initially overlooked its functionality. - Hardware Requirements for Mistral Deployment:
@cadavreuxinquired about the necessary hardware for deploying Mistral.@ved_ikkecommented that it may require 48GB RAM, while@egueesuggested that the 4-bit version might need about 21GB.@vhariationalprovided further details from the Mistral and HuggingFace documentation, recommending 2 A100 GPUs for the original version and indicating that a single A100 GPU is sufficient for the 4-bit quantized version.
▷ #ref-implem (4 messages):
- Mistral API for Inference and Prompt Templating:
@energetic_sparrow_28915asked whether they need to format messages according to the template suggested by Mistral when using the Mistral API for inference. They also asked about the format of the model when using the HF chat template. - In Response to Mistral API and Templating:
@vhariationalclarified that while using the API, one does not need to explicitly pass special tokens as it is handled internally like other LLM-as-API services. The chat template applied by the transformers library follows the documentation provided by Mistral. The provided links were Mistral's Documentation and an example on Hugging Face. - Regarding BOS Token:
@energetic_sparrow_28915mentioned that the official documentation mentions a space after the BOS token which the HF template does not provide, and asked for clarification. - Linking to Previous Discussion:
@vhariationalpointed old discussion in the same channel that addressed the question on the BOS token. The reference snippet from a previous conversation was shared.
▷ #finetuning (5 messages):
- Tokenizer Pad Token Solution: User
@krissayrosefound a workaround for a tokenizer problem by settingtokenizer.pad_token = '[PAD]'instead of usingtokenizer.eos_token. They expressed interest in hearing about potential better solutions. - Freezing Experts in Mistral 7b Finetuning: User
@alexworteegainquired about the possibility of freezing all experts except two while finetuning Mistral 7b. - Perspectives on Freezing Experts:
@energetic_sparrow_28915suggested that freezing particular MLP layers might not be the best approach to fine-tuning, implying that it could reduce the benefits of using an MoE (Mixture of Experts) model. - Benefits of Using Few Experts:
@alexworteegareiterated that, according to st moe and switch principles, using only a few experts doesn't lead to a significant quality drop. - Clarification on MoE Usage:
@energetic_sparrow_28915explained that the idea behind MoE is to utilize a sparse network to learn, meaning only a few experts are used at any one time, determined by a routing/gating mechanism.
▷ #showcase (2 messages):
- Open Source Version of OpenAI Assistants API: User
@louis030195shared about their work on an open source version of OpenAI Assistants API. This version allows users to switch to open source LLMs by just changing one line of code. Main benefits highlighted were: 100% data privacy, up to 75% cheaper, and up to 23x higher throughput. It supports features like retrieval, function calling and code interpreter, and can be used with Mistral LLM or any other such models. He also shared a link to the GitHub repository for the project (GitHub - stellar-amenities/assistants) and invited contributions to this open-source project with a mention that the codebase is in Rust while examples are usually in JS. He aims to make all prompts customizable and copying OpenAI API design with minimal hard coding.
▷ #random (9 messages🔥):
- Misinterpretation of Mistral AI Website:
@ben_arwdshared a humorous anecdote about his CMO having difficulty understanding the Mistral AI website, suggesting the platform's complexity.@mrobinocommented that Mistral is primarily targeted at developers and isn't designed for the average person. - Desire for Prestige:
@tonic_1joked about desiring an orange name on the server to show off on Saturday nights. - Company Progress Discussion:
@titaux12defended the progress of a young company that generates, although small, income within the first six months. - Queries about GPU Core Differences for Mistral and Transformers Usage:
@pier1337asked about the difference between using 16 and 19 GPU cores on a Mac Silicon for running Mistral and transformers, and also inquired about any available benchmarking against Nvidia.
▷ #la-plateforme (22 messages🔥):
-
Availability of the Plateforme:
@dafrenchfrogasked for information regarding the availability of Mistral's AI platform for new clients.@lerelaconfirmed that access would be granted soon. -
API Rate Limit Issue:
@flopsy1reported encountering an API rate limit error despite believing they used less than the allowed tokens.@lerelaclarified that this error could also occur if the account exceeds its current usage limit and offered to investigate further if provided with their Profile ID. -
Building Mixtral Connector:
@tonic_1announced plans to develop a Mixtral connector on their favourite orchestration libraries, hinting at the possibility of creating Mixtral swarms in the near future. They invited interested parties to join an event on their Discord server. -
Chat Format for System Messages:
@sekstiniqueried about the chat format for system messages and provided links to related Hugging Face configurations.@sekstiniand@nionedalso referenced the LLaMA-2 format and a Reddit thread discussing the effectiveness of different chat templates. -
Performance of Mistral-Small Model:
@sublimatorniqshared their experience with the Mistral-small model, noting varying results when tasked with summarizing around 5000 tokens. They couldn't locate the context lengths for each model and reported successful API calls. Other users discussed possible finetuning issues and techniques to improve model performance.
HuggingFace Discord Discord Summary
- Active dialogue surrounding AI models on smaller hardware, with examples like phi-2 and Mixtral, reinforcing the need for advancements due to hardware limitations.
- Several technical issues noted including: Runtime error NCCL with the 2x A10G GPU configuration while fine-tuning models; questions on handling HTML tags in product descriptions with embeddings (referencing all-MiniLM-L6-v2); questions about memory management and storage pertaining to LLM model training.
- Discussion on **social nuances of AI interactions, particularly gender nuances in bot responses, highlighting an awareness of public interest in this aspect of AI.
onceabeginneris learning Unet for image generation andhoang12345navigates the creative and worrying facets of AI after exploring a TED talk by Tom Graham titled 'The Incredible Creativity of Deepfakes — and the Worrying Future of AI'.- Busted out discussions on the inclusion of certain resources in the modern_nlp_2 GitHub repository and embedding life-events, as per an approach discussed in this Nature article.
- Creations shared include a song made by
@.bigdookieintegrating guitar, beatbox, and music continuations from MusicGen, a suggestion for an addition to the "awesome-repos.md" by@cloudhu, and an update from the Manifold Research group by@thebigbigbuddha(Sidh from Manifold). - Queries in
#[diffusion-discussions]pertained to diffusion models' applications such as modifying face images and outpainting, with references to the huggingface/diffusers GitHub repository. - Inquiry by
@noamgatabout resolving the issue of adding a token to already decoded token streams in O(1) time with the tokenizer. The bloodline of the query lies in the performance concern about handling edge cases like Unicode sequences and new word spaces in#[NLP].
HuggingFace Discord Channel Summaries
▷ #general (57 messages🔥🔥):
- Running Models on Smaller Hardware:
@vipitisin a series of messages discussed advancements in making AI models run on smaller hardware with notable solutions being quantization and availability of models with smaller parameter count that perform well on benchmarks. They mentioned models like phi-2 and Mixtral as examples and how hardware limitations have necessitated such advancements. However, the user did note that still some hardware is needed. - Question on Training with Multi-GPU Configuration: User
@blahblah6407reported a Runtime error NCCL when trying to fine-tune their model with 2x A10G GPUs, it had previously worked fine on single A100 GPU. They were looking for suggestions to fix the issue. - Query on Interacting with Bots:
@steamy_noodleshad a question about female bot responses, which implies that public is also discussing social nuances of AI interactions in the chat. - Discussion on the Use of HTML Tags in Embeddings:
@swastikkasked whether HTML tags in product description would improve the quality of the embeddings, while using all-MiniLM-L6-v2. - Training LLM Models and Memory: User
@lookingforspeedhad several queries related to training LLM models, including how memory works for the models, how the trained models are stored and if they would be lost in power outage scenarios.
▷ #today-im-learning (3 messages):
- Unet for Image Generation:
onceabeginnershared that they are learning how to use Unet for image generation. - The Creativity of Deepfakes and the Future of AI:
hoang12345expressed shock and concern after coming across a TED talk titled 'The Incredible Creativity of Deepfakes — and the Worrying Future of AI' by Tom Graham. The video can be found here.
▷ #cool-finds (3 messages):
- Contribution to modern_nlp_2 Repository:
@cloudhusuggested adding to the awesome-repos.md file in the modern_nlp_2 GitHub repository. No further details about the addition were mentioned. - Embeddings of Life-Events:
@toronelloshared a Nature article discussing an approach that creates embeddings of life-events in a single vector space. This approach allows for the prediction of different life-event outcomes. - Pin Request:
@tonic_1requested that someone pin a message. However, the specific message to pin was not specified.
▷ #i-made-this (5 messages):
-
Introductory Lecture on Image Classification:
@sumitsr71shared a link to an introductory lecture on Image Classification. The lecture covers topics like nearest neighbor classifier, k-nearest neighbor classifier, validation sets for hyperparameter tuning, amongst others. -
Modern NLP Repository:
@cloudhusuggested to add a certain resource to the "awesome-repos.md" file in the@andysingal'sModern NLP GitHub repository. -
Manifold Research Group Update:
@thebigbigbuddha(Sidh from Manifold) shared the recent progress made by the Manifold Research Group in their Research Log. They are working on massively multimodal "generalist" models and recruiting for a new project focused on developing state-of-the-art open source models that operate on GUI interfaces and APIs. -
Alve_om Status:
@om7059shared a link to Alve_om's status, however, no further information about the content of the status was provided. -
AI-Generated Music:
@.bigdookiecreated a song using guitar, beatbox, and music continuations from MusicGen. The entire creation process was done in one night.
▷ #diffusion-discussions (5 messages):
- Creating Dataset of Faces with Accessories Using Diffusion Models:
@blvrxdnthnhvqueried about a diffusion model that could add hats or helmets to a dataset of faces. They stated their current stable diffusion inpainting approach was not preserving facial details. - Question on Meta-Learning:
@bouncingsealoooasked about whether a particular training process could be considered meta-learning. Specifically, they described a scenario involving pretraining a model on a meta dataset using MAML, then further training the model on a target dataset and achieving improved results. - Resource for Face Modification:
@asrielhanshared a resource (Ziqiao Ma's tweet) where the demo video featured the addition of a hat to a face image. - ControlNet Inpainting Query:
@abstruse9851posted a question about the type of conditioning image to use when performing controlnet inpainting with the models"lllyasviel/control_v11p_sd15_inpaint"and"runwayml/stable-diffusion-inpainting". - Help Needed with Outpainting:
@d222097_requested advice on using the sd/sd-inpainting model for outpainting, sharing an open issue about the topic from the huggingface/diffusers GitHub repository. They were uncertain about the model input during training and the lack of constraints on the unknown areas.
▷ #NLP (3 messages):
- Performance Problem with Decoding Long Token Streams: User
@noamgatraised a concern over the decoding of long token streams, specifically on finding a way to add a token to an already decoded token stream in O(1) time. Their interest lies in the tokenizer, not the whole model. Edge cases like Unicode sequences and new word spaces must be considered.@vipitisfurther clarified whether the decoding concern involved the entirety of the model or just the tokenizer.
▷ #diffusion-discussions (5 messages):
- Diffusion Model for Modified Face Images:
@blvrxdnthnhvasked for a diffusion model that could superimpose hats or helmets onto existing face images. They tried using Stable Diffusion inpainting but it didn't retain the face details properly. - Meta Learning VS Pretraining:
@bouncingsealoooraised a query to differentiate between meta-learning and pretraining. They asked whether a model initially trained on a meta dataset using the MAML method for 50 epochs, and then further trained on target data for 600 epochs, giving a better result than a model without MAML could be considered a meta-learning application or not. - ControlNet Model Conditioning:
@abstruse9851had questions about the type of conditioning image to use with the ControlNet model "lllyasviel/control_v11p_sd15_inpaint" used in conjunction with the main model "runwayml/stable-diffusion-inpainting". - Outpainting with sd/sd-inpainting model:
@d222097_asked for assistance concerning outpainting with the sd/sd-inpainting model. They were unsure about the input of the model during the training stage.
LangChain AI Discord Summary
- SQL Virtual Database: User
@alewe5sought community assistance on working with a virtual database in SQL. - Task Planning with LLM:
@sampson7786had queries about the application of LLM for task planning in a specific business domain. - Writing Prompts Query:
@kelp710enquired about best practices for writing prompts, especially for non-English languages. - LangChain and Blockchain: Relation and possible partnerships between LangChain and Blockchain was explored by
@emreweb3,@lhc1921pointed out to open source contribution methods. - Pinecone Database and LLM issues:
@kingkkaktusand@nakhlarafidiscussed issues with Pinecone database queries and token limitations in LLM for large data respectively. - Ollama Halluciation Problem: User
@baller.1337reported an anomaly with Ollama hallucinating with provided code. - LangChain Agents vs Llama Index Agent Architecture: Differences and personal experiences with these architectures were queried by
@.psychickoala. - Streaming Chat Response Integration: User
@pradeepvj97sought help with integrating and fixing issues with the streaming chat response that interfaces with PDFs. - Manifold Research Group Updates: Shared by
@thebigbigbuddha, detailed updates on the group's progress, a link to their research log, recruitment notice for a novel project, and resources were shared - Research Log, Discord Channel, GitHub resources. - Langchain Expression Language (LCEL) Walkthrough: A video tutorial of LCEL was shared by
@merkleon YouTube, offering insight into its operation, and listing its advantages and disadvantages.
LangChain AI Channel Summaries
▷ #general (31 messages🔥):
- Working with a virtual database in SQL:
@alewe5is looking for feedback on working with a virtual database in SQL as they are encountering issues. - Using LLM for task planning:
@sampson7786asked about using LLM for task planning in a specific business domain. - Question about writing prompts:
@kelp710wanted to know whether it is more accurate to write prompts in English and have them return in their own language, or write the prompts directly in their own language. - LangChain and Blockchain partnerships:
@emreweb3inquired if LangChain, an open source project, was open to blockchain partnerships.@lhc1921suggested code contribution through a Pull Request for specific features. - Issues with Pinecone database:
@kingkkaktusasked for help while trying to query their Pinecone database and shared the code they are using which was throwing an error. - Token limitation problem for large data in LLM:
@nakhlarafi, a new user to LLM and LangChain, brought up the challenge of token limitation for processing large data.@kingkkaktussuggested using a vector database like Pinecone. - Hallucination problem with Ollama:
@baller.1337reported an issue with Ollama always hallucinating and shared their code. - Experience with LangChain agents vs Llama index agent architecture:
@.psychickoalaasked about the community's experiences with the different agent architectures and queried about adding a previous context/memory.
▷ #langchain-templates (1 messages):
- Streaming Chat Response Integration: User
@pradeepvj97shared a code snippet of their attempt to integrate a streaming chat response with their existing code which interfaces with PDFs, usingChatOpenAIand conversation entities in memory. However, they expressed facing issues with streaming tokens and sought help fixing it.
▷ #share-your-work (2 messages):
- Manifold Research Group Updates:
@thebigbigbuddhafrom Manifold shared the recent progress made by the group in their research log which can be found here. The group is also recruiting for a new project that focuses on State-of-the-art(SoTA) OS models for GUI interfaces and APIs usage, encouraging interested individuals to get involved via their Discord channel or check out the work on their Github. - Langchain Expression Language (LCEL) Walkthrough:
@merkleshared a YouTube video offering a walkthrough of the Langchain Expression Language (LCEL), providing insight into its usage and offering pros and cons for its implementation.
LLM Perf Enthusiasts AI Discord Summary
- Users enquired about the type of agent abstractions in use, including the LangChain vs. the Lllama index.
- OpenRouter received high praise as a recommendation due to its effectiveness.
- Features and utilization of LM Studio were highlighted, praising its ease in searching, downloading models and exposing chat API endpoints.
- The concept of hybrid retrieval using BM25 was proposed, with recommended improvements shared in a blog post.
- Strategies on efficient document extraction using gpt-4-turbo were discussed, with proposals for separating the extraction process into two different prompts for generic and document-specific fields, given that there's no parent-child dependency. Feedback provided suggested either keeping the two sets of data fields together or separating them based on their logical relationship.
LLM Perf Enthusiasts AI Channel Summaries
▷ #general (1 messages):
.psychickoala: which agent abstractions are you guys using? the langchain or the lllama index ones?
▷ #opensource (5 messages):
-
OpenRouter Recommendation:
@joshcho_shared his high opinion of OpenRouter, calling it "great". -
LM Studio Experience:
@joshcho_shared his positive experience of using LM Studio, finding it extremely convenient for handling local models. -
LM Studio Features: He continued to highlight LM Studio's features such as ease of searching and downloading models and even the ability to expose chat API endpoints, which he described as "wild".
▷ #rag (1 messages):
- Hybrid Retrieval Using BM25:
@evan_04487raised a question about the practice of using BM25 without subwords in hybrid retrieval processes, and highlighted that the LangChain BM25 retriever does not lowercase by default. They shared a blog post detailing simple improvements that can enhance BM25 implementation without extra LLM calls needed.
▷ #prompting (4 messages):
- Document Extraction via gpt-4-turbo: User
@pantsforbirdsshared their experience working on document extraction with gpt-4-turbo. They observed great results, however, they raised concern about the extraction of numerous JSON fields potentially affecting performance. - Two Sets of Fields:
@pantsforbirdsexplained that their extraction involved two sets of fields - a shared one across all documents and another set specific to the type of document. - Separate Extraction Prompts: In view of optimizing performance,
@pantsforbirdsproposed separating the extraction process into two different prompts: one for generic fields, and another for document-specific fields. - Lack of Parent-Child Dependency between Fields: They clarified that while the two sets of fields are related, there isn't a direct parent-child dependency.
- Feedback on the Idea of Separation:
@justahveesuggested that if there is a logical relationship between the two sets of data fields, they should be kept together; if not, it's better to separate the extraction process, since the first part of the generation could influence the latter in undesired ways.
Latent Space Discord Summary
- The community questioned and discussed access to Mixtral, indicating that several companies are competing in this market, as stated by
@swyxio. @swyxiohighlighted Etched's recent soft launch and the visualization they created for Mixtral, providing a reference to Kevin A. Fischer's tweet.- A new fine-tuning of LLAMA2 focusing on cybersecurity was announced as informed by
@swyxio, referring to a tweet by Miguel Tissera. - The community expressed concerns over apparent manipulation of the HuggingFace leaderboard, with calls for transparency on modifying models and 'the increasingly discussed UNA (Unique Neuron Attribution)', as mentioned by
@swyxioand supported by a link to the HuggingFace discussion.
Latent Space Channel Summaries
▷ #ai-general-chat (6 messages):
- Access to Mixtral:
@btdubbinsraised a question regarding how members of the community are accessing Mixtral, asking if anyone was using Anyscale calls. - Competitive Mixtral Market:
@swyxiopointed out that there are multiple companies, around seven, competing to provide Mixtral, all undercutting each other's prices and services. - Etched's Visuals for Mixtral:
@swyxioshared that the company Etched, which offers Mixtral, had soft-launched a few days ago and drew attention to Kevin A. Fischer's tweet featuring one of their notable visualizations for Mixtral: https://fxtwitter.com/kevinafischer/status/1736893685940605436?s=46&t=90xQ8sGy63D2OtiaoGJuww. - LLAMA2 Finetuning for Cybersecurity:
@swyxioindicated that a new fine-tuning of LLAMA2 dedicated to cybersecurity was unveiled, as showcased in a tweet by Miguel Tissera: https://fxtwitter.com/migtissera/status/1736946751440085320?s=46&t=90xQ8sGy63D2OtiaoGJuww. - Issues with HuggingFace Leaderboard:
@swyxioelaborated on the problem of the HuggingFace leaderboard being manipulated. An anonymous user claimed that people were taking the leading Mistral model, adding an unknown DPO to it, and, as a result, scoring higher than any Llama 70b. Concerns were raised about leaderboard credibility and calls were made for transparency, especially regarding the increasingly discussed UNA (Unique Neuron Attribution): https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard/discussions/444.
▷ #llm-paper-club (2 messages):
There were no topics, discussion points, or links/blogposts of interest to be summarized from the provided Discord chatbot messages.
Skunkworks AI Discord Summary
- Inquiry about the training data of Mistralic with a notable positive feedback on its performance, stating that "...it is one of the best."
- Discussion on integrating a vision model with the Segment Anything model for the purpose of image grounding. A user requests practical advice on how to achieve this.
- Recommendation to check out the LLaVA Plus and LLaVA Interactive projects, which make use of the Grounded-Segment-Anything repository for Visual Question Answering and Grounding.
- Cautionary note about potential difficulties in implementing aforementioned projects due to previously encountered issues.
Skunkworks AI Channel Summaries
▷ #general (1 messages):
- Mistralic Dataset Inquiry:
@spirobelasked about the datasets on which Mistralic was trained on. They also expressed a positive opinion on it's performance stating that "...it is one of the best."
▷ #bakklava-1 (3 messages):
- Merging Vision Model with Segment Anything Model for Image Grounding:
@occupying_marsrequested suggestions for merging a vision model with Segment Anything model to identify objects and their coordinates. - LLaVA Plus and LLaVA Interactive Projects:
@mrfoorecommended looking into the LLaVA Plus and LLaVA Interactive projects. The projects utilize the Grounded-Segment-Anything repository for Visual Question Answering and Grounding. - Issues with Project Implementation:
@mrfooalso mentioned that the aforementioned projects might be difficult to implement, citing issues encountered a few weeks prior.
DiscoResearch Discord Summary
-
Discussion on router learning in the training process for Mixtral.
@thootonexplains the function and importance of top-k in training, highlighting 'The gradients are propagated through the top-k experts and router embeddings, so the router does still learn (albeit not all of its params at once)'. -
Mention of the Megablocks project by
@propback. He shared a link to the project on GitHub, inviting contributions for its adaptation for fine-tuning with respect to Mixtral implementation. -
An unrelated comment from
leecigin the general channel questioning the purpose of the channel, indicating some confusion about its function or purpose.
DiscoResearch Channel Summaries
▷ #mixtral_implementation (2 messages):
-
Router Learning:
@thootonmentions the use of topk in the training process for Mixtral. He explains that "The gradients are propagated through the top-k experts and router embeddings, so the router does still learn (albeit not all of its params at once)". -
Megablocks Project:
@propbackshared a link to the Megablocks project on GitHub, inviting participants to help in adapting it for fine-tuning, specifying the Mixtral implementation.
▷ #general (1 messages):
leecig: I forgot why I'm here. What is this place?
AI Engineer Foundation Discord Summary
Only 1 channel had activity, so no need to summarize...
- AI Adoption by Business: User
@juanredshas initiated a discussion on increasing productivity and reducing costs in businesses by developing AI-based apps and agents. They asked the community for suggestions related to any possible frameworks. The discussion thread can be found here.
MLOps @Chipro Discord Summary
Only 1 channel had activity, so no need to summarize...
erisianrite: Please move to <#885998868200828928>
The Alignment Lab AI Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.
The YAIG (a16z Infra) Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.